Jul 06 22:22:05.956737 ip-10-0-133-105 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 06 22:22:05.956751 ip-10-0-133-105 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 06 22:22:05.956760 ip-10-0-133-105 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 06 22:22:05.957092 ip-10-0-133-105 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 06 22:22:16.180033 ip-10-0-133-105 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 06 22:22:16.180052 ip-10-0-133-105 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 2ad3f46454594e51ad07541402b211b8 --
Jul 06 22:23:53.064471 ip-10-0-133-105 systemd[1]: Starting Kubernetes Kubelet...
Jul 06 22:23:53.497292 ip-10-0-133-105 kubenswrapper[2563]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 06 22:23:53.497292 ip-10-0-133-105 kubenswrapper[2563]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 06 22:23:53.497292 ip-10-0-133-105 kubenswrapper[2563]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 06 22:23:53.497292 ip-10-0-133-105 kubenswrapper[2563]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 06 22:23:53.497292 ip-10-0-133-105 kubenswrapper[2563]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 06 22:23:53.499891 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.499799    2563 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 06 22:23:53.506400 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.506377    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 06 22:23:53.506400 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.506399    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 06 22:23:53.506400 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.506403    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 06 22:23:53.506400 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.506407    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 06 22:23:53.506565 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.506411    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 06 22:23:53.506565 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.506414    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 06 22:23:53.506565 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.506417    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 06 22:23:53.506565 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.506420    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 06 22:23:53.506565 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.506423    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 06 22:23:53.506565 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.506426    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 06 22:23:53.506565 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.506429    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 06 22:23:53.506565 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.506432    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 06 22:23:53.506565 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.506435    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 06 22:23:53.506565 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.506437    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 06 22:23:53.506565 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.506443    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 06 22:23:53.506565 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.506446    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 06 22:23:53.506565 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.506448    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 06 22:23:53.506565 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.506451    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 06 22:23:53.506565 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.506454    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 06 22:23:53.506565 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.506458    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 06 22:23:53.506565 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.506461    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 06 22:23:53.506565 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.506463    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 06 22:23:53.506565 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.506466    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 06 22:23:53.506565 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.506469    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 06 22:23:53.507024 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.506471    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 06 22:23:53.507024 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.506474    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 06 22:23:53.507024 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.506476    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 06 22:23:53.507024 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.506481    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 06 22:23:53.507024 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.506484    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 06 22:23:53.507024 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.506487    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 06 22:23:53.507024 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.506491    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 06 22:23:53.507024 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.506494    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 06 22:23:53.507024 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.506496    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 06 22:23:53.507024 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.506499    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 06 22:23:53.507024 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.506501    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 06 22:23:53.507024 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.506504    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 06 22:23:53.507024 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.506508    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 06 22:23:53.507024 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.506511    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 06 22:23:53.507024 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.506514    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 06 22:23:53.507024 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.506520    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 06 22:23:53.507024 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.506523    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 06 22:23:53.507024 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.506526    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 06 22:23:53.507024 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.506528    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 06 22:23:53.507024 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.506531    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 06 22:23:53.507553 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.506534    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 06 22:23:53.507553 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.506536    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 06 22:23:53.507553 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.506540    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 06 22:23:53.507553 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.506555    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 06 22:23:53.507553 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.506560    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 06 22:23:53.507553 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.506563    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 06 22:23:53.507553 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.506566    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 06 22:23:53.507553 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.506572    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 06 22:23:53.507553 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.506575    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 06 22:23:53.507553 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.506578    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 06 22:23:53.507553 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.506581    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 06 22:23:53.507553 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.506583    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 06 22:23:53.507553 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.506586    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 06 22:23:53.507553 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.506588    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 06 22:23:53.507553 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.506591    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 06 22:23:53.507553 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.506594    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 06 22:23:53.507553 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.506596    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 06 22:23:53.507553 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.506599    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 06 22:23:53.507553 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.506602    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 06 22:23:53.508101 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.506608    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 06 22:23:53.508101 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.506611    2563 feature_gate.go:328] unrecognized feature gate: Example2
Jul 06 22:23:53.508101 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.506614    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 06 22:23:53.508101 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.506617    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 06 22:23:53.508101 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.506619    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 06 22:23:53.508101 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.506622    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 06 22:23:53.508101 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.506626    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 06 22:23:53.508101 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.506629    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 06 22:23:53.508101 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.506632    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 06 22:23:53.508101 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.506634    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 06 22:23:53.508101 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.506637    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 06 22:23:53.508101 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.506642    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 06 22:23:53.508101 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.506647    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 06 22:23:53.508101 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.506650    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 06 22:23:53.508101 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.506652    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 06 22:23:53.508101 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.506655    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 06 22:23:53.508101 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.506658    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 06 22:23:53.508101 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.506661    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 06 22:23:53.508101 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.506664    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 06 22:23:53.508101 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.506667    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 06 22:23:53.508626 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.506669    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 06 22:23:53.508626 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.506672    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 06 22:23:53.508626 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.506675    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 06 22:23:53.508626 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.506678    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 06 22:23:53.508626 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.506683    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 06 22:23:53.508626 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.506685    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 06 22:23:53.508626 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.506688    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 06 22:23:53.508626 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.506690    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 06 22:23:53.508626 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.506693    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 06 22:23:53.508626 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.506696    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 06 22:23:53.508626 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.506699    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 06 22:23:53.508626 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.506702    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 06 22:23:53.508626 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.506704    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 06 22:23:53.508626 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.506708    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 06 22:23:53.508626 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.506712    2563 feature_gate.go:328] unrecognized feature gate: Example
Jul 06 22:23:53.508626 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.506715    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 06 22:23:53.508999 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.508894    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 06 22:23:53.508999 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.508906    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 06 22:23:53.508999 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.508910    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 06 22:23:53.508999 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.508913    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 06 22:23:53.508999 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.508916    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 06 22:23:53.508999 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.508920    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 06 22:23:53.508999 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.508922    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 06 22:23:53.508999 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.508925    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 06 22:23:53.508999 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.508929    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 06 22:23:53.508999 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.508931    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 06 22:23:53.508999 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.508934    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 06 22:23:53.508999 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.508936    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 06 22:23:53.508999 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.508939    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 06 22:23:53.508999 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.508941    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 06 22:23:53.508999 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.508944    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 06 22:23:53.508999 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.508947    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 06 22:23:53.508999 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.508950    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 06 22:23:53.508999 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.508952    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 06 22:23:53.508999 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.508955    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 06 22:23:53.508999 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.508957    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 06 22:23:53.509506 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.508959    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 06 22:23:53.509506 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.508962    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 06 22:23:53.509506 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.508965    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 06 22:23:53.509506 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.508968    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 06 22:23:53.509506 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.508970    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 06 22:23:53.509506 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.508973    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 06 22:23:53.509506 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.508975    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 06 22:23:53.509506 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.508977    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 06 22:23:53.509506 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.508980    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 06 22:23:53.509506 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.508982    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 06 22:23:53.509506 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.508984    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 06 22:23:53.509506 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.508987    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 06 22:23:53.509506 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.508989    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 06 22:23:53.509506 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.508992    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 06 22:23:53.509506 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.508995    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 06 22:23:53.509506 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.508998    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 06 22:23:53.509506 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509000    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 06 22:23:53.509506 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509003    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 06 22:23:53.509506 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509006    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 06 22:23:53.509985 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509009    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 06 22:23:53.509985 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509012    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 06 22:23:53.509985 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509014    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 06 22:23:53.509985 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509017    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 06 22:23:53.509985 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509019    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 06 22:23:53.509985 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509022    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 06 22:23:53.509985 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509024    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 06 22:23:53.509985 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509027    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 06 22:23:53.509985 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509029    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 06 22:23:53.509985 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509032    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 06 22:23:53.509985 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509036    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 06 22:23:53.509985 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509040    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 06 22:23:53.509985 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509043    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 06 22:23:53.509985 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509046    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 06 22:23:53.509985 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509049    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 06 22:23:53.509985 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509051    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 06 22:23:53.509985 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509054    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 06 22:23:53.509985 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509056    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 06 22:23:53.509985 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509058    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 06 22:23:53.510468 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509062    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 06 22:23:53.510468 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509064    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 06 22:23:53.510468 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509067    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 06 22:23:53.510468 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509072    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 06 22:23:53.510468 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509075    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 06 22:23:53.510468 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509077    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 06 22:23:53.510468 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509080    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 06 22:23:53.510468 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509082    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 06 22:23:53.510468 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509085    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 06 22:23:53.510468 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509088    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 06 22:23:53.510468 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509090    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 06 22:23:53.510468 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509093    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 06 22:23:53.510468 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509095    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 06 22:23:53.510468 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509097    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 06 22:23:53.510468 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509100    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 06 22:23:53.510468 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509103    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 06 22:23:53.510468 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509105    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 06 22:23:53.510468 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509109    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 06 22:23:53.510468 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509112    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 06 22:23:53.510934 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509116    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 06 22:23:53.510934 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509120    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 06 22:23:53.510934 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509123    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 06 22:23:53.510934 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509126    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 06 22:23:53.510934 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509129    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 06 22:23:53.510934 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509132    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 06 22:23:53.510934 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509135    2563 feature_gate.go:328] unrecognized feature gate: Example
Jul 06 22:23:53.510934 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509138    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 06 22:23:53.510934 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509141    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 06 22:23:53.510934 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509145    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 06 22:23:53.510934 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509147    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 06 22:23:53.510934 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509150    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 06 22:23:53.510934 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509152    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 06 22:23:53.510934 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509155    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 06 22:23:53.510934 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509157    2563 feature_gate.go:328] unrecognized feature gate: Example2
Jul 06 22:23:53.510934 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509160    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 06 22:23:53.510934 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509164    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 06 22:23:53.510934 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509182    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 06 22:23:53.510934 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509185    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 06 22:23:53.510934 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509187    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 06 22:23:53.511430 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509190    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 06 22:23:53.511430 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509192    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 06 22:23:53.511430 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509592    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 06 22:23:53.511430 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509597    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 06 22:23:53.511430 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509600    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 06 22:23:53.511430 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509603    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 06 22:23:53.511430 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509605    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 06 22:23:53.511430 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509608    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 06 22:23:53.511430 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509611    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 06 22:23:53.511430 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509613    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 06 22:23:53.511430 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509615    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 06 22:23:53.511430 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509618    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 06 22:23:53.511430 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509621    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 06 22:23:53.511430 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509624    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 06 22:23:53.511430 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509628    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 06 22:23:53.511430 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509631    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 06 22:23:53.511430 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509635    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 06 22:23:53.511430 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509637    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 06 22:23:53.511430 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509641    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 06 22:23:53.511875 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509644    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 06 22:23:53.511875 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509647    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 06 22:23:53.511875 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509649    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 06 22:23:53.511875 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509652    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 06 22:23:53.511875 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509655    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 06 22:23:53.511875 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509658    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 06 22:23:53.511875 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509660    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 06 22:23:53.511875 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509662    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 06 22:23:53.511875 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509665    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 06 22:23:53.511875 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509668    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 06 22:23:53.511875 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509670    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 06 22:23:53.511875 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509673    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 06 22:23:53.511875 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509675    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 06 22:23:53.511875 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509678    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 06 22:23:53.511875 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509680    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 06 22:23:53.511875 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509682    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 06 22:23:53.511875 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509685    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 06 22:23:53.511875 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509687    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 06 22:23:53.511875 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509690    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 06 22:23:53.511875 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509692    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 06 22:23:53.512352 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509695    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 06 22:23:53.512352 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509697    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 06 22:23:53.512352 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509699    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 06 22:23:53.512352 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509702    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 06 22:23:53.512352 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509705    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 06 22:23:53.512352 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509708    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 06 22:23:53.512352 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509710    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 06 22:23:53.512352 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509713    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 06 22:23:53.512352 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509715    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 06 22:23:53.512352 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509717    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 06 22:23:53.512352 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509720    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 06 22:23:53.512352 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509722    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 06 22:23:53.512352 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509725    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 06 22:23:53.512352 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509728    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 06 22:23:53.512352 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509730    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 06 22:23:53.512352 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509732    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 06 22:23:53.512352 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509735    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 06 22:23:53.512352 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509737    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 06 22:23:53.512352 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509739    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 06 22:23:53.512352 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509742    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 06 22:23:53.512823 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509744    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 06 22:23:53.512823 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509748    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 06 22:23:53.512823 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509752    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 06 22:23:53.512823 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509756    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 06 22:23:53.512823 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509760    2563 feature_gate.go:328] unrecognized feature gate: Example2
Jul 06 22:23:53.512823 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509763    2563 feature_gate.go:328] unrecognized feature gate: Example
Jul 06 22:23:53.512823 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509766    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 06 22:23:53.512823 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509770    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 06 22:23:53.512823 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509773    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 06 22:23:53.512823 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509776    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 06 22:23:53.512823 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509779    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 06 22:23:53.512823 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509782    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 06 22:23:53.512823 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509784    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 06 22:23:53.512823 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509787    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 06 22:23:53.512823 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509790    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 06 22:23:53.512823 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509793    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 06 22:23:53.512823 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509795    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 06 22:23:53.512823 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509798    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 06 22:23:53.512823 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509800    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 06 22:23:53.512823 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509802    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 06 22:23:53.513377 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509805    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 06 22:23:53.513377 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509807    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 06 22:23:53.513377 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509809    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 06 22:23:53.513377 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509812    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 06 22:23:53.513377 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509815    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 06 22:23:53.513377 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509818    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 06 22:23:53.513377 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509820    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 06 22:23:53.513377 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509823    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 06 22:23:53.513377 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509825    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 06 22:23:53.513377 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509827    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 06 22:23:53.513377 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509830    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 06 22:23:53.513377 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509832    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 06 22:23:53.513377 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509835    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 06 22:23:53.513377 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509837    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 06 22:23:53.513377 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509840    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 06 22:23:53.513377 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509842    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 06 22:23:53.513377 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509845    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 06 22:23:53.513377 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509847    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 06 22:23:53.513377 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509850    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 06 22:23:53.513826 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509852    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 06 22:23:53.513826 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509855    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 06 22:23:53.513826 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.509857    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 06 22:23:53.513826 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.509945    2563 flags.go:64] FLAG: --address="0.0.0.0"
Jul 06 22:23:53.513826 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.509955    2563 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 06 22:23:53.513826 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.509963    2563 flags.go:64] FLAG: --anonymous-auth="true"
Jul 06 22:23:53.513826 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.509969    2563 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 06 22:23:53.513826 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.509975    2563 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 06 22:23:53.513826 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.509979    2563 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 06 22:23:53.513826 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.509983    2563 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 06 22:23:53.513826 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.509987    2563 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 06 22:23:53.513826 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.509990    2563 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 06 22:23:53.513826 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.509993    2563 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 06 22:23:53.513826 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.509997    2563 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 06 22:23:53.513826 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.510000    2563 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 06 22:23:53.513826 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.510003    2563 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 06 22:23:53.513826 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.510006    2563 flags.go:64] FLAG: --cgroup-root=""
Jul 06 22:23:53.513826 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.510009    2563 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 06 22:23:53.513826 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.510012    2563 flags.go:64] FLAG: --client-ca-file=""
Jul 06 22:23:53.513826 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.510016    2563 flags.go:64] FLAG: --cloud-provider="external"
Jul 06 22:23:53.513826 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.510019    2563 flags.go:64] FLAG: --cluster-dns="[]"
Jul 06 22:23:53.513826 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.510023    2563 flags.go:64] FLAG: --cluster-domain=""
Jul 06 22:23:53.513826 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.510026    2563 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 06 22:23:53.513826 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.510029    2563 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 06 22:23:53.514414 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.510032    2563 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 06 22:23:53.514414 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.510035    2563 flags.go:64] FLAG: --container-log-max-files="5"
Jul 06 22:23:53.514414 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.510046    2563 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 06 22:23:53.514414 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.510049    2563 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 06 22:23:53.514414 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.510052    2563 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 06 22:23:53.514414 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.510055    2563 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 06 22:23:53.514414 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.510058    2563 flags.go:64] FLAG: --contention-profiling="false"
Jul 06 22:23:53.514414 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.510061    2563 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 06 22:23:53.514414 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.510063    2563 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 06 22:23:53.514414 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.510066    2563 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 06 22:23:53.514414 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.510069    2563 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 06 22:23:53.514414 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.510074    2563 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 06 22:23:53.514414 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.510076    2563 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 06 22:23:53.514414 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.510079    2563 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 06 22:23:53.514414 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.510082    2563 flags.go:64] FLAG: --enable-load-reader="false"
Jul 06 22:23:53.514414 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.510085    2563 flags.go:64] FLAG: --enable-server="true"
Jul 06 22:23:53.514414 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.510088    2563 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 06 22:23:53.514414 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.510092    2563 flags.go:64] FLAG: --event-burst="100"
Jul 06 22:23:53.514414 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.510095    2563 flags.go:64] FLAG: --event-qps="50"
Jul 06 22:23:53.514414 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.510097    2563 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 06 22:23:53.514414 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.510100    2563 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 06 22:23:53.514414 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.510103    2563 flags.go:64] FLAG: --eviction-hard=""
Jul 06 22:23:53.514414 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.510107    2563 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 06 22:23:53.514414 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.510109    2563 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 06 22:23:53.514981 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.510114    2563 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 06 22:23:53.514981 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.510117    2563 flags.go:64] FLAG: --eviction-soft=""
Jul 06 22:23:53.514981 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.510120    2563 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 06 22:23:53.514981 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.510123    2563 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 06 22:23:53.514981 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.510127    2563 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 06 22:23:53.514981 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.510129    2563 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 06 22:23:53.514981 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.510132    2563 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 06 22:23:53.514981 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.510135    2563 flags.go:64] FLAG: --fail-swap-on="true"
Jul 06 22:23:53.514981 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.510138    2563 flags.go:64] FLAG: --feature-gates=""
Jul 06 22:23:53.514981 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.510141    2563 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 06 22:23:53.514981 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.510145    2563 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 06 22:23:53.514981 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.510148    2563 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 06 22:23:53.514981 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.510151    2563 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 06 22:23:53.514981 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.510154    2563 flags.go:64] FLAG: --healthz-port="10248"
Jul 06 22:23:53.514981 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.510157    2563 flags.go:64] FLAG: --help="false"
Jul 06 22:23:53.514981 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.510160    2563 flags.go:64] FLAG: --hostname-override="ip-10-0-133-105.ec2.internal"
Jul 06 22:23:53.514981 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.510163    2563 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 06 22:23:53.514981 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.510178    2563 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 06 22:23:53.514981 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.510182    2563 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 06 22:23:53.514981 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.510186    2563 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 06 22:23:53.514981 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.510189    2563 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 06 22:23:53.514981 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.510192    2563 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 06 22:23:53.514981 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.510195    2563 flags.go:64] FLAG: --image-service-endpoint=""
Jul 06 22:23:53.515566 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.510197    2563 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 06 22:23:53.515566 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.510200    2563 flags.go:64] FLAG: --kube-api-burst="100"
Jul 06 22:23:53.515566 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.510203    2563 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 06 22:23:53.515566 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.510206    2563 flags.go:64] FLAG: --kube-api-qps="50"
Jul 06 22:23:53.515566 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.510209    2563 flags.go:64] FLAG: --kube-reserved=""
Jul 06 22:23:53.515566 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.510212    2563 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 06 22:23:53.515566 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.510214    2563 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 06 22:23:53.515566 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.510217    2563 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 06 22:23:53.515566 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.510220    2563 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 06 22:23:53.515566 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.510224    2563 flags.go:64] FLAG: --lock-file=""
Jul 06 22:23:53.515566 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.510227    2563 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 06 22:23:53.515566 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.510230    2563 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 06 22:23:53.515566 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.510233    2563 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 06 22:23:53.515566 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.510238    2563 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 06 22:23:53.515566 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.510241    2563 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 06 22:23:53.515566 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.510244    2563 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 06 22:23:53.515566 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.510247    2563 flags.go:64] FLAG: --logging-format="text"
Jul 06 22:23:53.515566 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.510250    2563 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 06 22:23:53.515566 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.510253    2563 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 06 22:23:53.515566 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.510257    2563 flags.go:64] FLAG: --manifest-url=""
Jul 06 22:23:53.515566 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.510260    2563 flags.go:64] FLAG: --manifest-url-header=""
Jul 06 22:23:53.515566 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.510265    2563 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 06 22:23:53.515566 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.510268    2563 flags.go:64] FLAG: --max-open-files="1000000"
Jul 06 22:23:53.515566 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.510272    2563 flags.go:64] FLAG: --max-pods="110"
Jul 06 22:23:53.515566 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.510275    2563 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 06 22:23:53.516177 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.510277    2563 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 06 22:23:53.516177 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.510280    2563 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 06 22:23:53.516177 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.510283    2563 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 06 22:23:53.516177 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.510286    2563 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 06 22:23:53.516177 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.510289    2563 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 06 22:23:53.516177 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.510292    2563 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 06 22:23:53.516177 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.510299    2563 flags.go:64] FLAG: --node-status-max-images="50"
Jul 06 22:23:53.516177 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.510302    2563 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 06 22:23:53.516177 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.510306    2563 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 06 22:23:53.516177 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.510309    2563 flags.go:64] FLAG: --pod-cidr=""
Jul 06 22:23:53.516177 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.510312    2563 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5"
Jul 06 22:23:53.516177 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.510317    2563 flags.go:64] FLAG: --pod-manifest-path=""
Jul 06 22:23:53.516177 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.510320    2563 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 06 22:23:53.516177 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.510323    2563 flags.go:64] FLAG: --pods-per-core="0"
Jul 06 22:23:53.516177 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.510325    2563 flags.go:64] FLAG: --port="10250"
Jul 06 22:23:53.516177 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.510328    2563 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 06 22:23:53.516177 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.510331    2563 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-07f9736363ac7dd6c"
Jul 06 22:23:53.516177 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.510336    2563 flags.go:64] FLAG: --qos-reserved=""
Jul 06 22:23:53.516177 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.510338    2563 flags.go:64] FLAG: --read-only-port="10255"
Jul 06 22:23:53.516177 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.510341    2563 flags.go:64] FLAG: --register-node="true"
Jul 06 22:23:53.516177 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.510344    2563 flags.go:64] FLAG: --register-with-taints=""
Jul 06 22:23:53.516177 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.510347    2563 flags.go:64] FLAG: --registry-burst="10"
Jul 06 22:23:53.516177 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.510351    2563 flags.go:64] FLAG: --registry-qps="5"
Jul 06 22:23:53.516177 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.510354    2563 flags.go:64] FLAG: --reserved-cpus=""
Jul 06 22:23:53.516791 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.510357    2563 flags.go:64] FLAG: --reserved-memory=""
Jul 06 22:23:53.516791 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.510360    2563 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 06 22:23:53.516791 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.510364    2563 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 06 22:23:53.516791 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.510367    2563 flags.go:64] FLAG: --rotate-certificates="false"
Jul 06 22:23:53.516791 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.510370    2563 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 06 22:23:53.516791 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.510373    2563 flags.go:64] FLAG: --runonce="false"
Jul 06 22:23:53.516791 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.510376    2563 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 06 22:23:53.516791 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.510379    2563 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 06 22:23:53.516791 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.510382    2563 flags.go:64] FLAG: --seccomp-default="false"
Jul 06 22:23:53.516791 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.510385    2563 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 06 22:23:53.516791 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.510388    2563 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 06 22:23:53.516791 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.510390    2563 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 06 22:23:53.516791 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.510393    2563 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 06 22:23:53.516791 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.510396    2563 flags.go:64] FLAG: --storage-driver-password="root"
Jul 06 22:23:53.516791 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.510399    2563 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 06 22:23:53.516791 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.510402    2563 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 06 22:23:53.516791 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.510405    2563 flags.go:64] FLAG: --storage-driver-user="root"
Jul 06 22:23:53.516791 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.510407    2563 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 06 22:23:53.516791 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.510411    2563 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 06 22:23:53.516791 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.510414    2563 flags.go:64] FLAG: --system-cgroups=""
Jul 06 22:23:53.516791 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.510417    2563 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 06 22:23:53.516791 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.510422    2563 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 06 22:23:53.516791 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.510425    2563 flags.go:64] FLAG: --tls-cert-file=""
Jul 06 22:23:53.516791 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.510428    2563 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 06 22:23:53.516791 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.510432    2563 flags.go:64] FLAG: --tls-min-version=""
Jul 06 22:23:53.517435 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.510436    2563 flags.go:64] FLAG: --tls-private-key-file=""
Jul 06 22:23:53.517435 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.510439    2563 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 06 22:23:53.517435 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.510442    2563 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 06 22:23:53.517435 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.510445    2563 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 06 22:23:53.517435 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.510448    2563 flags.go:64] FLAG: --v="2"
Jul 06 22:23:53.517435 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.510459    2563 flags.go:64] FLAG: --version="false"
Jul 06 22:23:53.517435 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.510464    2563 flags.go:64] FLAG: --vmodule=""
Jul 06 22:23:53.517435 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.510468    2563 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 06 22:23:53.517435 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.510471    2563 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 06 22:23:53.517435 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.510622    2563 feature_gate.go:328] unrecognized feature gate: Example
Jul 06 22:23:53.517435 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.510627    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 06 22:23:53.517435 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.510630    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 06 22:23:53.517435 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.510632    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 06 22:23:53.517435 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.510635    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 06 22:23:53.517435 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.510638    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 06 22:23:53.517435 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.510641    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 06 22:23:53.517435 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.510643    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 06 22:23:53.517435 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.510646    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 06 22:23:53.517435 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.510648    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 06 22:23:53.517435 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.510651    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 06 22:23:53.517435 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.510653    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 06 22:23:53.517950 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.510657    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 06 22:23:53.517950 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.510661    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 06 22:23:53.517950 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.510664    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 06 22:23:53.517950 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.510667    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 06 22:23:53.517950 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.510670    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 06 22:23:53.517950 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.510672    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 06 22:23:53.517950 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.510675    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 06 22:23:53.517950 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.510677    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 06 22:23:53.517950 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.510679    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 06 22:23:53.517950 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.510682    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 06 22:23:53.517950 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.510684    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 06 22:23:53.517950 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.510688    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 06 22:23:53.517950 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.510690    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 06 22:23:53.517950 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.510693    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 06 22:23:53.517950 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.510695    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 06 22:23:53.517950 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.510698    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 06 22:23:53.517950 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.510700    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 06 22:23:53.517950 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.510703    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 06 22:23:53.517950 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.510706    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 06 22:23:53.517950 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.510708    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 06 22:23:53.518463 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.510714    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 06 22:23:53.518463 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.510716    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 06 22:23:53.518463 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.510719    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 06 22:23:53.518463 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.510721    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 06 22:23:53.518463 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.510724    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 06 22:23:53.518463 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.510726    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 06 22:23:53.518463 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.510729    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 06 22:23:53.518463 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.510732    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 06 22:23:53.518463 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.510736    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 06 22:23:53.518463 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.510739    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 06 22:23:53.518463 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.510741    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 06 22:23:53.518463 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.510744    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 06 22:23:53.518463 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.510746    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 06 22:23:53.518463 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.510749    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 06 22:23:53.518463 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.510751    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 06 22:23:53.518463 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.510753    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 06 22:23:53.518463 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.510756    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 06 22:23:53.518463 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.510758    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 06 22:23:53.518463 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.510761    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 06 22:23:53.518919 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.510763    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 06 22:23:53.518919 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.510766    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 06 22:23:53.518919 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.510769    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 06 22:23:53.518919 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.510771    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 06 22:23:53.518919 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.510775    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 06 22:23:53.518919 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.510777    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 06 22:23:53.518919 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.510780    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 06 22:23:53.518919 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.510782    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 06 22:23:53.518919 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.510785    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 06 22:23:53.518919 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.510788    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 06 22:23:53.518919 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.510790    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 06 22:23:53.518919 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.510793    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 06 22:23:53.518919 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.510796    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 06 22:23:53.518919 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.510801    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 06 22:23:53.518919 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.510803    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 06 22:23:53.518919 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.510806    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 06 22:23:53.518919 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.510808    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 06 22:23:53.518919 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.510811    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 06 22:23:53.518919 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.510813    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 06 22:23:53.518919 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.510816    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 06 22:23:53.519421 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.510819    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 06 22:23:53.519421 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.510821    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 06 22:23:53.519421 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.510824    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 06 22:23:53.519421 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.510826    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 06 22:23:53.519421 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.510829    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 06 22:23:53.519421 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.510832    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 06 22:23:53.519421 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.510834    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 06 22:23:53.519421 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.510836    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 06 22:23:53.519421 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.510839    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 06 22:23:53.519421 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.510841    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 06 22:23:53.519421 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.510845    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 06 22:23:53.519421 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.510849    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 06 22:23:53.519421 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.510851    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 06 22:23:53.519421 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.510854    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 06 22:23:53.519421 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.510857    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 06 22:23:53.519421 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.510859    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 06 22:23:53.519421 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.510863    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 06 22:23:53.519421 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.510865    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 06 22:23:53.519421 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.510868    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 06 22:23:53.519421 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.510870    2563 feature_gate.go:328] unrecognized feature gate: Example2
Jul 06 22:23:53.519910 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.510873    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 06 22:23:53.519910 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.510875    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 06 22:23:53.519910 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.510878    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 06 22:23:53.519910 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.510880    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 06 22:23:53.519910 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.510883    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 06 22:23:53.519910 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.510887    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 06 22:23:53.519910 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.510890    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 06 22:23:53.519910 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.510892    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 06 22:23:53.519910 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.511483    2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 06 22:23:53.519910 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.518671    2563 server.go:529] "Kubelet version" kubeletVersion="v1.34.8"
Jul 06 22:23:53.519910 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.518685    2563 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 06 22:23:53.519910 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.518704    2563 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 06 22:23:53.519910 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.518709    2563 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 06 22:23:53.519910 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.518774    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 06 22:23:53.520273 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.518779    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 06 22:23:53.520273 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.518782    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 06 22:23:53.520273 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.518785    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 06 22:23:53.520273 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.518788    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 06 22:23:53.520273 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.518790    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 06 22:23:53.520273 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.518794    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 06 22:23:53.520273 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.518796    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 06 22:23:53.520273 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.518799    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 06 22:23:53.520273 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.518801    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 06 22:23:53.520273 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.518803    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 06 22:23:53.520273 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.518806    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 06 22:23:53.520273 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.518809    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 06 22:23:53.520273 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.518811    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 06 22:23:53.520273 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.518814    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 06 22:23:53.520273 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.518816    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 06 22:23:53.520273 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.518818    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 06 22:23:53.520273 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.518821    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 06 22:23:53.520273 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.518823    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 06 22:23:53.520273 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.518826    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 06 22:23:53.520273 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.518828    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 06 22:23:53.520750 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.518830    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 06 22:23:53.520750 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.518833    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 06 22:23:53.520750 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.518835    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 06 22:23:53.520750 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.518839    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 06 22:23:53.520750 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.518842    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 06 22:23:53.520750 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.518844    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 06 22:23:53.520750 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.518847    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 06 22:23:53.520750 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.518849    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 06 22:23:53.520750 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.518852    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 06 22:23:53.520750 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.518856    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 06 22:23:53.520750 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.518860    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 06 22:23:53.520750 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.518863    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 06 22:23:53.520750 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.518866    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 06 22:23:53.520750 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.518869    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 06 22:23:53.520750 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.518871    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 06 22:23:53.520750 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.518874    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 06 22:23:53.520750 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.518877    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 06 22:23:53.520750 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.518879    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 06 22:23:53.520750 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.518882    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 06 22:23:53.521206 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.518884    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 06 22:23:53.521206 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.518887    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 06 22:23:53.521206 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.518889    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 06 22:23:53.521206 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.518892    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 06 22:23:53.521206 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.518894    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 06 22:23:53.521206 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.518897    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 06 22:23:53.521206 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.518899    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 06 22:23:53.521206 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.518902    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 06 22:23:53.521206 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.518905    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 06 22:23:53.521206 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.518907    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 06 22:23:53.521206 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.518910    2563 feature_gate.go:328] unrecognized feature gate: Example2
Jul 06 22:23:53.521206 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.518912    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 06 22:23:53.521206 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.518914    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 06 22:23:53.521206 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.518917    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 06 22:23:53.521206 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.518920    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 06 22:23:53.521206 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.518922    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 06 22:23:53.521206 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.518925    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 06 22:23:53.521206 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.518928    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 06 22:23:53.521206 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.518931    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 06 22:23:53.521206 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.518934    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 06 22:23:53.521685 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.518937    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 06 22:23:53.521685 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.518939    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 06 22:23:53.521685 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.518942    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 06 22:23:53.521685 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.518945    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 06 22:23:53.521685 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.518947    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 06 22:23:53.521685 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.518950    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 06 22:23:53.521685 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.518952    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 06 22:23:53.521685 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.518955    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 06 22:23:53.521685 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.518957    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 06 22:23:53.521685 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.518959    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 06 22:23:53.521685 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.518962    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 06 22:23:53.521685 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.518964    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 06 22:23:53.521685 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.518966    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 06 22:23:53.521685 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.518969    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 06 22:23:53.521685 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.518972    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 06 22:23:53.521685 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.518974    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 06 22:23:53.521685 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.518976    2563 feature_gate.go:328] unrecognized feature gate: Example
Jul 06 22:23:53.521685 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.518979    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 06 22:23:53.521685 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.518981    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 06 22:23:53.521685 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.518983    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 06 22:23:53.522215 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.518986    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 06 22:23:53.522215 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.518988    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 06 22:23:53.522215 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.518991    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 06 22:23:53.522215 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.518993    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 06 22:23:53.522215 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.518995    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 06 22:23:53.522215 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.518998    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 06 22:23:53.522215 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.519000    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 06 22:23:53.522215 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.519002    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 06 22:23:53.522215 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.519005    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 06 22:23:53.522215 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.519007    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 06 22:23:53.522215 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.519014    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 06 22:23:53.522215 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.519018    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 06 22:23:53.522215 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.519021    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 06 22:23:53.522215 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.519024    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 06 22:23:53.522215 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.519027    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 06 22:23:53.522215 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.519031    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 06 22:23:53.522215 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.519034    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 06 22:23:53.522215 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.519037    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 06 22:23:53.522215 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.519039    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 06 22:23:53.522677 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.519044    2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 06 22:23:53.522677 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.519152    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 06 22:23:53.522677 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.519156    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 06 22:23:53.522677 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.519159    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 06 22:23:53.522677 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.519161    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 06 22:23:53.522677 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.519181    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 06 22:23:53.522677 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.519185    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 06 22:23:53.522677 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.519188    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 06 22:23:53.522677 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.519192    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 06 22:23:53.522677 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.519195    2563 feature_gate.go:328] unrecognized feature gate: Example2
Jul 06 22:23:53.522677 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.519198    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 06 22:23:53.522677 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.519201    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 06 22:23:53.522677 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.519203    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 06 22:23:53.522677 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.519206    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 06 22:23:53.522677 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.519208    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 06 22:23:53.523048 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.519211    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 06 22:23:53.523048 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.519213    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 06 22:23:53.523048 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.519215    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 06 22:23:53.523048 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.519218    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 06 22:23:53.523048 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.519221    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 06 22:23:53.523048 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.519223    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 06 22:23:53.523048 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.519225    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 06 22:23:53.523048 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.519228    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 06 22:23:53.523048 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.519230    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 06 22:23:53.523048 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.519233    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 06 22:23:53.523048 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.519236    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 06 22:23:53.523048 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.519239    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 06 22:23:53.523048 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.519242    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 06 22:23:53.523048 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.519244    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 06 22:23:53.523048 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.519247    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 06 22:23:53.523048 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.519249    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 06 22:23:53.523048 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.519252    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 06 22:23:53.523048 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.519254    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 06 22:23:53.523048 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.519257    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 06 22:23:53.523048 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.519259    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 06 22:23:53.523528 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.519261    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 06 22:23:53.523528 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.519264    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 06 22:23:53.523528 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.519266    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 06 22:23:53.523528 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.519269    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 06 22:23:53.523528 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.519271    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 06 22:23:53.523528 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.519274    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 06 22:23:53.523528 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.519276    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 06 22:23:53.523528 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.519279    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 06 22:23:53.523528 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.519281    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 06 22:23:53.523528 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.519284    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 06 22:23:53.523528 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.519286    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 06 22:23:53.523528 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.519289    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 06 22:23:53.523528 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.519291    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 06 22:23:53.523528 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.519293    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 06 22:23:53.523528 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.519296    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 06 22:23:53.523528 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.519298    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 06 22:23:53.523528 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.519300    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 06 22:23:53.523528 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.519303    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 06 22:23:53.523528 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.519305    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 06 22:23:53.523982 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.519307    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 06 22:23:53.523982 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.519309    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 06 22:23:53.523982 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.519312    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 06 22:23:53.523982 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.519316    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 06 22:23:53.523982 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.519318    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 06 22:23:53.523982 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.519321    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 06 22:23:53.523982 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.519325    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 06 22:23:53.523982 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.519328    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 06 22:23:53.523982 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.519330    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 06 22:23:53.523982 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.519333    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 06 22:23:53.523982 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.519336    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 06 22:23:53.523982 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.519338    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 06 22:23:53.523982 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.519340    2563 feature_gate.go:328] unrecognized feature gate: Example
Jul 06 22:23:53.523982 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.519343    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 06 22:23:53.523982 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.519345    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 06 22:23:53.523982 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.519348    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 06 22:23:53.523982 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.519350    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 06 22:23:53.523982 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.519353    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 06 22:23:53.523982 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.519355    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 06 22:23:53.524450 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.519358    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 06 22:23:53.524450 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.519360    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 06 22:23:53.524450 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.519362    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 06 22:23:53.524450 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.519365    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 06 22:23:53.524450 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.519367    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 06 22:23:53.524450 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.519369    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 06 22:23:53.524450 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.519372    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 06 22:23:53.524450 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.519374    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 06 22:23:53.524450 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.519377    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 06 22:23:53.524450 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.519379    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 06 22:23:53.524450 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.519381    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 06 22:23:53.524450 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.519384    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 06 22:23:53.524450 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.519386    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 06 22:23:53.524450 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.519388    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 06 22:23:53.524450 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.519392    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 06 22:23:53.524450 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.519395    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 06 22:23:53.524450 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.519398    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 06 22:23:53.524450 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.519401    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 06 22:23:53.524450 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.519404    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 06 22:23:53.524450 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.519407    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 06 22:23:53.524926 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.519409    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 06 22:23:53.524926 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.519412    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 06 22:23:53.524926 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.519415    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 06 22:23:53.524926 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.519418    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 06 22:23:53.524926 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.519420    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 06 22:23:53.524926 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.519422    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 06 22:23:53.524926 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:53.519425    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 06 22:23:53.524926 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.519430    2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 06 22:23:53.524926 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.519534    2563 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 06 22:23:53.525680 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.525665    2563 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 06 22:23:53.527138 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.527125    2563 server.go:1019] "Starting client certificate rotation"
Jul 06 22:23:53.527245 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.527229    2563 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 06 22:23:53.527279 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.527269    2563 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 06 22:23:53.531992 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.531979    2563 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 06 22:23:53.533082 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.533062    2563 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 06 22:23:53.545218 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.545182    2563 log.go:25] "Validated CRI v1 runtime API"
Jul 06 22:23:53.548117 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.548103    2563 log.go:25] "Validated CRI v1 image API"
Jul 06 22:23:53.549231 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.549213    2563 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 06 22:23:53.551758 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.551739    2563 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 8f2511f5-fa6c-42f5-8eb8-a3aa3b52cf3d:/dev/nvme0n1p4 a4412eb6-fa59-425e-abd3-cf45b4fb1015:/dev/nvme0n1p3]
Jul 06 22:23:53.551820 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.551758    2563 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 06 22:23:53.556992 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.556969    2563 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 06 22:23:53.557751 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.557637    2563 manager.go:216] Machine: {Timestamp:2026-07-06 22:23:53.55529953 +0000 UTC m=+0.365346911 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3098898 MemoryCapacity:32812175360 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec254b44e5848140dbc45d757358d473 SystemUUID:ec254b44-e584-8140-dbc4-5d757358d473 BootID:2ad3f464-5459-4e51-ad07-541402b211b8 Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:71:0c:91:c8:d5 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:71:0c:91:c8:d5 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:ba:dc:16:db:aa:bc Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812175360 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 06 22:23:53.557751 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.557750    2563 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 06 22:23:53.557929 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.557913    2563 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 06 22:23:53.561322 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.561295    2563 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 06 22:23:53.561488 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.561328    2563 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-133-105.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 06 22:23:53.562097 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.562086    2563 topology_manager.go:138] "Creating topology manager with none policy"
Jul 06 22:23:53.562158 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.562102    2563 container_manager_linux.go:309] "Creating device plugin manager"
Jul 06 22:23:53.562158 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.562113    2563 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 06 22:23:53.562158 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.562132    2563 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 06 22:23:53.563637 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.563625    2563 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 06 22:23:53.567139 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.567125    2563 state_mem.go:36] "Initialized new in-memory state store"
Jul 06 22:23:53.567282 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.567271    2563 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 06 22:23:53.567403 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.567309    2563 kubelet.go:486] "Attempting to sync node with API server"
Jul 06 22:23:53.567403 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.567322    2563 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 06 22:23:53.567403 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.567335    2563 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 06 22:23:53.567403 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.567348    2563 kubelet.go:398] "Adding apiserver pod source"
Jul 06 22:23:53.567403 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.567360    2563 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 06 22:23:53.567757 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.567738    2563 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 06 22:23:53.567828 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.567762    2563 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 06 22:23:53.570089 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.570065    2563 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1"
Jul 06 22:23:53.571985 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.571971    2563 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 06 22:23:53.572054 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.572002    2563 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 06 22:23:53.573100 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.573089    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 06 22:23:53.573154 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.573104    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 06 22:23:53.573154 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.573110    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 06 22:23:53.573154 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.573116    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 06 22:23:53.573154 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.573125    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 06 22:23:53.573154 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.573130    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 06 22:23:53.573154 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.573138    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 06 22:23:53.573154 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.573143    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 06 22:23:53.573154 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.573150    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 06 22:23:53.573154 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.573155    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 06 22:23:53.573398 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.573163    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 06 22:23:53.573398 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.573185    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 06 22:23:53.573921 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.573907    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 06 22:23:53.573971 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.573927    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 06 22:23:53.577349 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.577333    2563 server.go:1268] "Started kubelet"
Jul 06 22:23:53.577424 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.577391    2563 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 06 22:23:53.577870 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.577645    2563 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 06 22:23:53.578061 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.578043    2563 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 06 22:23:53.578248 ip-10-0-133-105 systemd[1]: Started Kubernetes Kubelet.
Jul 06 22:23:53.578486 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.578465    2563 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-znfz6"
Jul 06 22:23:53.582233 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.579768    2563 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 06 22:23:53.582320 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.582228    2563 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-133-105.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 06 22:23:53.582579 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:23:53.582547    2563 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-133-105.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 06 22:23:53.583098 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:23:53.583070    2563 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 06 22:23:53.583894 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:23:53.583839    2563 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 06 22:23:53.583973 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.583910    2563 server.go:310] "Adding debug handlers to kubelet server"
Jul 06 22:23:53.585948 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.585930    2563 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-znfz6"
Jul 06 22:23:53.588437 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.588419    2563 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 06 22:23:53.588437 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.588428    2563 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 06 22:23:53.588584 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:23:53.588512    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-105.ec2.internal\" not found"
Jul 06 22:23:53.589380 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.589362    2563 factory.go:55] Registering systemd factory
Jul 06 22:23:53.589380 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.589383    2563 factory.go:223] Registration of the systemd container factory successfully
Jul 06 22:23:53.589725 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.589709    2563 factory.go:153] Registering CRI-O factory
Jul 06 22:23:53.589802 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.589728    2563 factory.go:223] Registration of the crio container factory successfully
Jul 06 22:23:53.589856 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.589800    2563 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 06 22:23:53.589856 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.589827    2563 factory.go:103] Registering Raw factory
Jul 06 22:23:53.589856 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.589843    2563 manager.go:1196] Started watching for new ooms in manager
Jul 06 22:23:53.590002 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:23:53.588771    2563 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-105.ec2.internal.18bfd399791b4b2d  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-105.ec2.internal,UID:ip-10-0-133-105.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-133-105.ec2.internal,},FirstTimestamp:2026-07-06 22:23:53.577294637 +0000 UTC m=+0.387342018,LastTimestamp:2026-07-06 22:23:53.577294637 +0000 UTC m=+0.387342018,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-105.ec2.internal,}"
Jul 06 22:23:53.590002 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.589797    2563 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 06 22:23:53.590002 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.589876    2563 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 06 22:23:53.590002 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.589796    2563 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 06 22:23:53.590231 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.590024    2563 reconstruct.go:97] "Volume reconstruction finished"
Jul 06 22:23:53.590231 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.590034    2563 reconciler.go:29] "Reconciler: start to sync state"
Jul 06 22:23:53.590231 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.590211    2563 manager.go:349] Starting recovery of all containers
Jul 06 22:23:53.594557 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.594487    2563 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 06 22:23:53.597435 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:23:53.597412    2563 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-133-105.ec2.internal\" not found" node="ip-10-0-133-105.ec2.internal"
Jul 06 22:23:53.601972 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.601930    2563 manager.go:354] Recovery completed
Jul 06 22:23:53.605793 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.605783    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 06 22:23:53.607899 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.607882    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-105.ec2.internal" event="NodeHasSufficientMemory"
Jul 06 22:23:53.607986 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.607914    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-105.ec2.internal" event="NodeHasNoDiskPressure"
Jul 06 22:23:53.607986 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.607929    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-105.ec2.internal" event="NodeHasSufficientPID"
Jul 06 22:23:53.608588 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.608565    2563 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 06 22:23:53.608588 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.608586    2563 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 06 22:23:53.608717 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.608608    2563 state_mem.go:36] "Initialized new in-memory state store"
Jul 06 22:23:53.608717 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.608589    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 06 22:23:53.609454 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.609438    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-105.ec2.internal" event="NodeHasSufficientMemory"
Jul 06 22:23:53.609523 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.609471    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-105.ec2.internal" event="NodeHasNoDiskPressure"
Jul 06 22:23:53.609523 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.609488    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-105.ec2.internal" event="NodeHasSufficientPID"
Jul 06 22:23:53.609871 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.609856    2563 policy_none.go:49] "None policy: Start"
Jul 06 22:23:53.609919 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.609891    2563 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 06 22:23:53.609919 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.609902    2563 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 06 22:23:53.610860 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.610849    2563 policy_none.go:47] "Start"
Jul 06 22:23:53.658967 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.658788    2563 manager.go:337] "Starting Device Plugin manager"
Jul 06 22:23:53.659126 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:23:53.658998    2563 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 06 22:23:53.659126 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.659011    2563 server.go:87] "Starting device plugin registration server"
Jul 06 22:23:53.659333 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.659320    2563 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 06 22:23:53.659415 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.659336    2563 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 06 22:23:53.659464 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.659446    2563 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 06 22:23:53.659561 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.659540    2563 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 06 22:23:53.659561 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.659553    2563 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 06 22:23:53.660101 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:23:53.660079    2563 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 06 22:23:53.660234 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:23:53.660132    2563 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-133-105.ec2.internal\" not found"
Jul 06 22:23:53.687311 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.687274    2563 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 06 22:23:53.688656 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.688631    2563 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 06 22:23:53.688773 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.688673    2563 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 06 22:23:53.688773 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.688700    2563 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 06 22:23:53.688773 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:23:53.688739    2563 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 06 22:23:53.691057 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.691038    2563 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 06 22:23:53.759786 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.759681    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 06 22:23:53.760695 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.760676    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-105.ec2.internal" event="NodeHasSufficientMemory"
Jul 06 22:23:53.760804 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.760711    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-105.ec2.internal" event="NodeHasNoDiskPressure"
Jul 06 22:23:53.760804 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.760723    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-105.ec2.internal" event="NodeHasSufficientPID"
Jul 06 22:23:53.760804 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.760749    2563 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-133-105.ec2.internal"
Jul 06 22:23:53.769390 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.769369    2563 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-133-105.ec2.internal"
Jul 06 22:23:53.769480 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:23:53.769397    2563 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-133-105.ec2.internal\": node \"ip-10-0-133-105.ec2.internal\" not found"
Jul 06 22:23:53.788090 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:23:53.788059    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-105.ec2.internal\" not found"
Jul 06 22:23:53.789051 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.789030    2563 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-105.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-133-105.ec2.internal"]
Jul 06 22:23:53.789105 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.789096    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 06 22:23:53.790546 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.790528    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-105.ec2.internal" event="NodeHasSufficientMemory"
Jul 06 22:23:53.790608 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.790567    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-105.ec2.internal" event="NodeHasNoDiskPressure"
Jul 06 22:23:53.790608 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.790583    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-105.ec2.internal" event="NodeHasSufficientPID"
Jul 06 22:23:53.792051 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.792039    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 06 22:23:53.792211 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.792195    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-105.ec2.internal"
Jul 06 22:23:53.792260 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.792228    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 06 22:23:53.793571 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.793552    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-105.ec2.internal" event="NodeHasSufficientMemory"
Jul 06 22:23:53.793664 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.793588    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-105.ec2.internal" event="NodeHasNoDiskPressure"
Jul 06 22:23:53.793664 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.793601    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-105.ec2.internal" event="NodeHasSufficientPID"
Jul 06 22:23:53.794211 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.794195    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-105.ec2.internal" event="NodeHasSufficientMemory"
Jul 06 22:23:53.794294 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.794221    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-105.ec2.internal" event="NodeHasNoDiskPressure"
Jul 06 22:23:53.794294 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.794233    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-105.ec2.internal" event="NodeHasSufficientPID"
Jul 06 22:23:53.795502 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.795488    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-105.ec2.internal"
Jul 06 22:23:53.795570 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.795511    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 06 22:23:53.796161 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.796148    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-105.ec2.internal" event="NodeHasSufficientMemory"
Jul 06 22:23:53.796268 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.796187    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-105.ec2.internal" event="NodeHasNoDiskPressure"
Jul 06 22:23:53.796268 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.796197    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-105.ec2.internal" event="NodeHasSufficientPID"
Jul 06 22:23:53.816981 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:23:53.816953    2563 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-105.ec2.internal\" not found" node="ip-10-0-133-105.ec2.internal"
Jul 06 22:23:53.821570 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:23:53.821549    2563 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-105.ec2.internal\" not found" node="ip-10-0-133-105.ec2.internal"
Jul 06 22:23:53.888422 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:23:53.888398    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-105.ec2.internal\" not found"
Jul 06 22:23:53.890693 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.890640    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a7e777c6384fb2d7e87381cc5d382585-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-105.ec2.internal\" (UID: \"a7e777c6384fb2d7e87381cc5d382585\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-105.ec2.internal"
Jul 06 22:23:53.890748 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.890719    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a7e777c6384fb2d7e87381cc5d382585-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-105.ec2.internal\" (UID: \"a7e777c6384fb2d7e87381cc5d382585\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-105.ec2.internal"
Jul 06 22:23:53.989436 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:23:53.989386    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-105.ec2.internal\" not found"
Jul 06 22:23:53.991619 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.991601    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a7e777c6384fb2d7e87381cc5d382585-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-105.ec2.internal\" (UID: \"a7e777c6384fb2d7e87381cc5d382585\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-105.ec2.internal"
Jul 06 22:23:53.991713 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.991633    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a7e777c6384fb2d7e87381cc5d382585-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-105.ec2.internal\" (UID: \"a7e777c6384fb2d7e87381cc5d382585\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-105.ec2.internal"
Jul 06 22:23:53.991713 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.991661    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/80e55e8f9c22a5bcba25f5b76deba1dd-config\") pod \"kube-apiserver-proxy-ip-10-0-133-105.ec2.internal\" (UID: \"80e55e8f9c22a5bcba25f5b76deba1dd\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-105.ec2.internal"
Jul 06 22:23:53.991785 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.991721    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a7e777c6384fb2d7e87381cc5d382585-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-105.ec2.internal\" (UID: \"a7e777c6384fb2d7e87381cc5d382585\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-105.ec2.internal"
Jul 06 22:23:53.991785 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:53.991721    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a7e777c6384fb2d7e87381cc5d382585-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-105.ec2.internal\" (UID: \"a7e777c6384fb2d7e87381cc5d382585\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-105.ec2.internal"
Jul 06 22:23:54.090221 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:23:54.090131    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-105.ec2.internal\" not found"
Jul 06 22:23:54.092308 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.092285    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/80e55e8f9c22a5bcba25f5b76deba1dd-config\") pod \"kube-apiserver-proxy-ip-10-0-133-105.ec2.internal\" (UID: \"80e55e8f9c22a5bcba25f5b76deba1dd\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-105.ec2.internal"
Jul 06 22:23:54.092357 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.092338    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/80e55e8f9c22a5bcba25f5b76deba1dd-config\") pod \"kube-apiserver-proxy-ip-10-0-133-105.ec2.internal\" (UID: \"80e55e8f9c22a5bcba25f5b76deba1dd\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-105.ec2.internal"
Jul 06 22:23:54.120443 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.120419    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-105.ec2.internal"
Jul 06 22:23:54.125422 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.125404    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-105.ec2.internal"
Jul 06 22:23:54.190717 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:23:54.190667    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-105.ec2.internal\" not found"
Jul 06 22:23:54.291232 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:23:54.291203    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-105.ec2.internal\" not found"
Jul 06 22:23:54.391789 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:23:54.391697    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-105.ec2.internal\" not found"
Jul 06 22:23:54.492268 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:23:54.492238    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-105.ec2.internal\" not found"
Jul 06 22:23:54.496665 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.496645    2563 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 06 22:23:54.526522 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.526488    2563 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 06 22:23:54.526977 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.526642    2563 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 06 22:23:54.526977 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.526679    2563 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 06 22:23:54.526977 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.526687    2563 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 06 22:23:54.567841 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.567809    2563 apiserver.go:52] "Watching apiserver"
Jul 06 22:23:54.586921 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.586890    2563 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 06 22:23:54.587206 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.587188    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/node-resolver-tgnb8","openshift-multus/multus-vs62k","openshift-network-diagnostics/network-check-target-6qmpm","openshift-ovn-kubernetes/ovnkube-node-t6jpk","kube-system/global-pull-secret-syncer-mc48r","kube-system/konnectivity-agent-z88v4","openshift-image-registry/node-ca-9l9l5","openshift-multus/multus-additional-cni-plugins-rzszm","openshift-multus/network-metrics-daemon-bbjgn","openshift-network-operator/iptables-alerter-slmxp","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b8d5j","openshift-cluster-node-tuning-operator/tuned-8dl2h"]
Jul 06 22:23:54.588544 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.588527    2563 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 06 22:23:54.588651 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.588580    2563 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-105.ec2.internal"
Jul 06 22:23:54.588742 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.588726    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-tgnb8"
Jul 06 22:23:54.589921 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.589890    2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-05 22:18:54 +0000 UTC" deadline="2027-12-12 17:53:49.758353208 +0000 UTC"
Jul 06 22:23:54.589921 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.589919    2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12571h29m55.168438005s"
Jul 06 22:23:54.590092 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.589949    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-vs62k"
Jul 06 22:23:54.590852 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.590834    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-7dj6s\""
Jul 06 22:23:54.591095 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.591076    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 06 22:23:54.591207 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.591129    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6qmpm"
Jul 06 22:23:54.591282 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:23:54.591211    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6qmpm" podUID="da82c66a-1f8a-4315-8f45-8b449e2e46b9"
Jul 06 22:23:54.591608 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.591595    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 06 22:23:54.592763 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.592482    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 06 22:23:54.592763 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.592556    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-f54vl\""
Jul 06 22:23:54.592763 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.592593    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 06 22:23:54.592763 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.592595    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 06 22:23:54.592763 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.592708    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 06 22:23:54.593994 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.593976    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-t6jpk"
Jul 06 22:23:54.597365 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.595278    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/79685e2c-e9e0-4ebe-93e6-12dd50c3c75c-multus-cni-dir\") pod \"multus-vs62k\" (UID: \"79685e2c-e9e0-4ebe-93e6-12dd50c3c75c\") " pod="openshift-multus/multus-vs62k"
Jul 06 22:23:54.597365 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.595313    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/79685e2c-e9e0-4ebe-93e6-12dd50c3c75c-host-run-k8s-cni-cncf-io\") pod \"multus-vs62k\" (UID: \"79685e2c-e9e0-4ebe-93e6-12dd50c3c75c\") " pod="openshift-multus/multus-vs62k"
Jul 06 22:23:54.597365 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.595348    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/79685e2c-e9e0-4ebe-93e6-12dd50c3c75c-multus-daemon-config\") pod \"multus-vs62k\" (UID: \"79685e2c-e9e0-4ebe-93e6-12dd50c3c75c\") " pod="openshift-multus/multus-vs62k"
Jul 06 22:23:54.597365 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.595376    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fzmgw\" (UniqueName: \"kubernetes.io/projected/9c27bdd9-eca8-4ef2-a260-56bb21de6b1e-kube-api-access-fzmgw\") pod \"node-resolver-tgnb8\" (UID: \"9c27bdd9-eca8-4ef2-a260-56bb21de6b1e\") " pod="openshift-dns/node-resolver-tgnb8"
Jul 06 22:23:54.597365 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.595470    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/79685e2c-e9e0-4ebe-93e6-12dd50c3c75c-system-cni-dir\") pod \"multus-vs62k\" (UID: \"79685e2c-e9e0-4ebe-93e6-12dd50c3c75c\") " pod="openshift-multus/multus-vs62k"
Jul 06 22:23:54.597365 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.595503    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/79685e2c-e9e0-4ebe-93e6-12dd50c3c75c-cnibin\") pod \"multus-vs62k\" (UID: \"79685e2c-e9e0-4ebe-93e6-12dd50c3c75c\") " pod="openshift-multus/multus-vs62k"
Jul 06 22:23:54.597365 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.595529    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/79685e2c-e9e0-4ebe-93e6-12dd50c3c75c-cni-binary-copy\") pod \"multus-vs62k\" (UID: \"79685e2c-e9e0-4ebe-93e6-12dd50c3c75c\") " pod="openshift-multus/multus-vs62k"
Jul 06 22:23:54.597365 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.595606    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/79685e2c-e9e0-4ebe-93e6-12dd50c3c75c-host-var-lib-cni-bin\") pod \"multus-vs62k\" (UID: \"79685e2c-e9e0-4ebe-93e6-12dd50c3c75c\") " pod="openshift-multus/multus-vs62k"
Jul 06 22:23:54.597365 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.595677    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/79685e2c-e9e0-4ebe-93e6-12dd50c3c75c-hostroot\") pod \"multus-vs62k\" (UID: \"79685e2c-e9e0-4ebe-93e6-12dd50c3c75c\") " pod="openshift-multus/multus-vs62k"
Jul 06 22:23:54.597365 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.595711    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/79685e2c-e9e0-4ebe-93e6-12dd50c3c75c-host-run-multus-certs\") pod \"multus-vs62k\" (UID: \"79685e2c-e9e0-4ebe-93e6-12dd50c3c75c\") " pod="openshift-multus/multus-vs62k"
Jul 06 22:23:54.597365 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.595873    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8vk4k\" (UniqueName: \"kubernetes.io/projected/79685e2c-e9e0-4ebe-93e6-12dd50c3c75c-kube-api-access-8vk4k\") pod \"multus-vs62k\" (UID: \"79685e2c-e9e0-4ebe-93e6-12dd50c3c75c\") " pod="openshift-multus/multus-vs62k"
Jul 06 22:23:54.597365 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.595968    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/79685e2c-e9e0-4ebe-93e6-12dd50c3c75c-os-release\") pod \"multus-vs62k\" (UID: \"79685e2c-e9e0-4ebe-93e6-12dd50c3c75c\") " pod="openshift-multus/multus-vs62k"
Jul 06 22:23:54.597365 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.596181    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/79685e2c-e9e0-4ebe-93e6-12dd50c3c75c-host-run-netns\") pod \"multus-vs62k\" (UID: \"79685e2c-e9e0-4ebe-93e6-12dd50c3c75c\") " pod="openshift-multus/multus-vs62k"
Jul 06 22:23:54.597365 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.596214    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/79685e2c-e9e0-4ebe-93e6-12dd50c3c75c-host-var-lib-cni-multus\") pod \"multus-vs62k\" (UID: \"79685e2c-e9e0-4ebe-93e6-12dd50c3c75c\") " pod="openshift-multus/multus-vs62k"
Jul 06 22:23:54.597365 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.596297    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/79685e2c-e9e0-4ebe-93e6-12dd50c3c75c-host-var-lib-kubelet\") pod \"multus-vs62k\" (UID: \"79685e2c-e9e0-4ebe-93e6-12dd50c3c75c\") " pod="openshift-multus/multus-vs62k"
Jul 06 22:23:54.597365 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.596337    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/79685e2c-e9e0-4ebe-93e6-12dd50c3c75c-multus-conf-dir\") pod \"multus-vs62k\" (UID: \"79685e2c-e9e0-4ebe-93e6-12dd50c3c75c\") " pod="openshift-multus/multus-vs62k"
Jul 06 22:23:54.597365 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.596366    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/79685e2c-e9e0-4ebe-93e6-12dd50c3c75c-etc-kubernetes\") pod \"multus-vs62k\" (UID: \"79685e2c-e9e0-4ebe-93e6-12dd50c3c75c\") " pod="openshift-multus/multus-vs62k"
Jul 06 22:23:54.598401 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.596402    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gvmwj\" (UniqueName: \"kubernetes.io/projected/da82c66a-1f8a-4315-8f45-8b449e2e46b9-kube-api-access-gvmwj\") pod \"network-check-target-6qmpm\" (UID: \"da82c66a-1f8a-4315-8f45-8b449e2e46b9\") " pod="openshift-network-diagnostics/network-check-target-6qmpm"
Jul 06 22:23:54.598401 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.596451    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9c27bdd9-eca8-4ef2-a260-56bb21de6b1e-tmp-dir\") pod \"node-resolver-tgnb8\" (UID: \"9c27bdd9-eca8-4ef2-a260-56bb21de6b1e\") " pod="openshift-dns/node-resolver-tgnb8"
Jul 06 22:23:54.598401 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.596499    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/79685e2c-e9e0-4ebe-93e6-12dd50c3c75c-multus-socket-dir-parent\") pod \"multus-vs62k\" (UID: \"79685e2c-e9e0-4ebe-93e6-12dd50c3c75c\") " pod="openshift-multus/multus-vs62k"
Jul 06 22:23:54.598401 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.596527    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/9c27bdd9-eca8-4ef2-a260-56bb21de6b1e-hosts-file\") pod \"node-resolver-tgnb8\" (UID: \"9c27bdd9-eca8-4ef2-a260-56bb21de6b1e\") " pod="openshift-dns/node-resolver-tgnb8"
Jul 06 22:23:54.598401 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.596648    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 06 22:23:54.598401 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.596994    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 06 22:23:54.598401 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.597000    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-rx4cr\""
Jul 06 22:23:54.598401 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.598028    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 06 22:23:54.598401 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.598076    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 06 22:23:54.598401 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.598247    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mc48r"
Jul 06 22:23:54.598401 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:23:54.598303    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mc48r" podUID="aad76575-952c-4b85-b64a-2523529c1cf4"
Jul 06 22:23:54.598761 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.598547    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 06 22:23:54.598761 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.598557    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 06 22:23:54.599653 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.599640    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-z88v4"
Jul 06 22:23:54.600524 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.600510    2563 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 06 22:23:54.600638 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.600619    2563 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-105.ec2.internal"
Jul 06 22:23:54.600878 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.600864    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-9l9l5"
Jul 06 22:23:54.601663 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.601640    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 06 22:23:54.601907 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.601892    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 06 22:23:54.602003 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.601989    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-4dsk6\""
Jul 06 22:23:54.602658 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.602639    2563 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 06 22:23:54.602726 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.602657    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-rzszm"
Jul 06 22:23:54.603046 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.603027    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 06 22:23:54.603130 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.603090    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 06 22:23:54.603130 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.603106    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 06 22:23:54.603392 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.603376    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-npsdl\""
Jul 06 22:23:54.604156 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.604140    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bbjgn"
Jul 06 22:23:54.604279 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:23:54.604262    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bbjgn" podUID="69b0b78e-e934-433b-bfdb-5fd6d8ba4a29"
Jul 06 22:23:54.605405 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.605390    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-slmxp"
Jul 06 22:23:54.605503 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.605438    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 06 22:23:54.605781 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.605764    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 06 22:23:54.605870 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.605809    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-p7z6f\""
Jul 06 22:23:54.606849 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.606835    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b8d5j"
Jul 06 22:23:54.607476 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.607460    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 06 22:23:54.607562 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.607503    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 06 22:23:54.607725 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.607708    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 06 22:23:54.607784 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.607745    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-rrq5j\""
Jul 06 22:23:54.608311 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.608294    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-105.ec2.internal"]
Jul 06 22:23:54.608387 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.608376    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-8dl2h"
Jul 06 22:23:54.609287 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.609261    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 06 22:23:54.609287 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.609270    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-sbkd6\""
Jul 06 22:23:54.609568 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.609333    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 06 22:23:54.609568 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.609374    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 06 22:23:54.610775 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.610755    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 06 22:23:54.610855 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.610786    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 06 22:23:54.610855 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.610761    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-j9st7\""
Jul 06 22:23:54.611925 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.611909    2563 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 06 22:23:54.612023 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.611960    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-133-105.ec2.internal"]
Jul 06 22:23:54.622181 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.622155    2563 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-2t4xq"
Jul 06 22:23:54.632366 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.632348    2563 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-2t4xq"
Jul 06 22:23:54.690911 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.690888    2563 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 06 22:23:54.696767 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.696741    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/79685e2c-e9e0-4ebe-93e6-12dd50c3c75c-host-run-multus-certs\") pod \"multus-vs62k\" (UID: \"79685e2c-e9e0-4ebe-93e6-12dd50c3c75c\") " pod="openshift-multus/multus-vs62k"
Jul 06 22:23:54.696912 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.696781    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8vk4k\" (UniqueName: \"kubernetes.io/projected/79685e2c-e9e0-4ebe-93e6-12dd50c3c75c-kube-api-access-8vk4k\") pod \"multus-vs62k\" (UID: \"79685e2c-e9e0-4ebe-93e6-12dd50c3c75c\") " pod="openshift-multus/multus-vs62k"
Jul 06 22:23:54.696912 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.696808    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/03309f33-ff47-447d-956f-c89bdd4f8054-host-cni-netd\") pod \"ovnkube-node-t6jpk\" (UID: \"03309f33-ff47-447d-956f-c89bdd4f8054\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6jpk"
Jul 06 22:23:54.696912 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.696830    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/03309f33-ff47-447d-956f-c89bdd4f8054-ovnkube-script-lib\") pod \"ovnkube-node-t6jpk\" (UID: \"03309f33-ff47-447d-956f-c89bdd4f8054\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6jpk"
Jul 06 22:23:54.696912 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.696853    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/aad76575-952c-4b85-b64a-2523529c1cf4-kubelet-config\") pod \"global-pull-secret-syncer-mc48r\" (UID: \"aad76575-952c-4b85-b64a-2523529c1cf4\") " pod="kube-system/global-pull-secret-syncer-mc48r"
Jul 06 22:23:54.696912 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.696876    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/686a5812-1550-4164-8e13-080cf1953bdb-konnectivity-ca\") pod \"konnectivity-agent-z88v4\" (UID: \"686a5812-1550-4164-8e13-080cf1953bdb\") " pod="kube-system/konnectivity-agent-z88v4"
Jul 06 22:23:54.696912 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.696902    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/79685e2c-e9e0-4ebe-93e6-12dd50c3c75c-host-run-netns\") pod \"multus-vs62k\" (UID: \"79685e2c-e9e0-4ebe-93e6-12dd50c3c75c\") " pod="openshift-multus/multus-vs62k"
Jul 06 22:23:54.697200 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.696868    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/79685e2c-e9e0-4ebe-93e6-12dd50c3c75c-host-run-multus-certs\") pod \"multus-vs62k\" (UID: \"79685e2c-e9e0-4ebe-93e6-12dd50c3c75c\") " pod="openshift-multus/multus-vs62k"
Jul 06 22:23:54.697200 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.696941    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/79685e2c-e9e0-4ebe-93e6-12dd50c3c75c-etc-kubernetes\") pod \"multus-vs62k\" (UID: \"79685e2c-e9e0-4ebe-93e6-12dd50c3c75c\") " pod="openshift-multus/multus-vs62k"
Jul 06 22:23:54.697200 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.696977    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9c27bdd9-eca8-4ef2-a260-56bb21de6b1e-tmp-dir\") pod \"node-resolver-tgnb8\" (UID: \"9c27bdd9-eca8-4ef2-a260-56bb21de6b1e\") " pod="openshift-dns/node-resolver-tgnb8"
Jul 06 22:23:54.697200 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.696991    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/79685e2c-e9e0-4ebe-93e6-12dd50c3c75c-host-run-netns\") pod \"multus-vs62k\" (UID: \"79685e2c-e9e0-4ebe-93e6-12dd50c3c75c\") " pod="openshift-multus/multus-vs62k"
Jul 06 22:23:54.697200 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.697008    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/03309f33-ff47-447d-956f-c89bdd4f8054-host-slash\") pod \"ovnkube-node-t6jpk\" (UID: \"03309f33-ff47-447d-956f-c89bdd4f8054\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6jpk"
Jul 06 22:23:54.697200 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.697036    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ntg8q\" (UniqueName: \"kubernetes.io/projected/809882c8-1856-4dc4-ac3f-710954ae19c7-kube-api-access-ntg8q\") pod \"iptables-alerter-slmxp\" (UID: \"809882c8-1856-4dc4-ac3f-710954ae19c7\") " pod="openshift-network-operator/iptables-alerter-slmxp"
Jul 06 22:23:54.697200 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.697064    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/79685e2c-e9e0-4ebe-93e6-12dd50c3c75c-multus-socket-dir-parent\") pod \"multus-vs62k\" (UID: \"79685e2c-e9e0-4ebe-93e6-12dd50c3c75c\") " pod="openshift-multus/multus-vs62k"
Jul 06 22:23:54.697200 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.697088    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/03309f33-ff47-447d-956f-c89bdd4f8054-host-kubelet\") pod \"ovnkube-node-t6jpk\" (UID: \"03309f33-ff47-447d-956f-c89bdd4f8054\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6jpk"
Jul 06 22:23:54.697200 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.697121    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/55b0aff6-b51e-4948-8a6f-04eb6756ab6e-etc-sysctl-conf\") pod \"tuned-8dl2h\" (UID: \"55b0aff6-b51e-4948-8a6f-04eb6756ab6e\") " pod="openshift-cluster-node-tuning-operator/tuned-8dl2h"
Jul 06 22:23:54.697200 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.697145    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/79685e2c-e9e0-4ebe-93e6-12dd50c3c75c-etc-kubernetes\") pod \"multus-vs62k\" (UID: \"79685e2c-e9e0-4ebe-93e6-12dd50c3c75c\") " pod="openshift-multus/multus-vs62k"
Jul 06 22:23:54.697200 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.697145    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/55b0aff6-b51e-4948-8a6f-04eb6756ab6e-tmp\") pod \"tuned-8dl2h\" (UID: \"55b0aff6-b51e-4948-8a6f-04eb6756ab6e\") " pod="openshift-cluster-node-tuning-operator/tuned-8dl2h"
Jul 06 22:23:54.697698 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.697245    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/79685e2c-e9e0-4ebe-93e6-12dd50c3c75c-multus-socket-dir-parent\") pod \"multus-vs62k\" (UID: \"79685e2c-e9e0-4ebe-93e6-12dd50c3c75c\") " pod="openshift-multus/multus-vs62k"
Jul 06 22:23:54.697698 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.697317    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/03309f33-ff47-447d-956f-c89bdd4f8054-ovnkube-config\") pod \"ovnkube-node-t6jpk\" (UID: \"03309f33-ff47-447d-956f-c89bdd4f8054\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6jpk"
Jul 06 22:23:54.697698 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.697346    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dd78p\" (UniqueName: \"kubernetes.io/projected/6a3050e9-b393-4767-a9c2-dafdf2a17925-kube-api-access-dd78p\") pod \"node-ca-9l9l5\" (UID: \"6a3050e9-b393-4767-a9c2-dafdf2a17925\") " pod="openshift-image-registry/node-ca-9l9l5"
Jul 06 22:23:54.697698 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.697370    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9902a376-c450-4867-9c7c-d9e5300bce11-cni-binary-copy\") pod \"multus-additional-cni-plugins-rzszm\" (UID: \"9902a376-c450-4867-9c7c-d9e5300bce11\") " pod="openshift-multus/multus-additional-cni-plugins-rzszm"
Jul 06 22:23:54.697698 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.697395    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/55b0aff6-b51e-4948-8a6f-04eb6756ab6e-etc-sysctl-d\") pod \"tuned-8dl2h\" (UID: \"55b0aff6-b51e-4948-8a6f-04eb6756ab6e\") " pod="openshift-cluster-node-tuning-operator/tuned-8dl2h"
Jul 06 22:23:54.697698 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.697422    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/79685e2c-e9e0-4ebe-93e6-12dd50c3c75c-cni-binary-copy\") pod \"multus-vs62k\" (UID: \"79685e2c-e9e0-4ebe-93e6-12dd50c3c75c\") " pod="openshift-multus/multus-vs62k"
Jul 06 22:23:54.697698 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.697446    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/03309f33-ff47-447d-956f-c89bdd4f8054-host-run-netns\") pod \"ovnkube-node-t6jpk\" (UID: \"03309f33-ff47-447d-956f-c89bdd4f8054\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6jpk"
Jul 06 22:23:54.697698 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.697468    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9c27bdd9-eca8-4ef2-a260-56bb21de6b1e-tmp-dir\") pod \"node-resolver-tgnb8\" (UID: \"9c27bdd9-eca8-4ef2-a260-56bb21de6b1e\") " pod="openshift-dns/node-resolver-tgnb8"
Jul 06 22:23:54.697698 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.697470    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/03309f33-ff47-447d-956f-c89bdd4f8054-host-cni-bin\") pod \"ovnkube-node-t6jpk\" (UID: \"03309f33-ff47-447d-956f-c89bdd4f8054\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6jpk"
Jul 06 22:23:54.697698 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.697588    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e5e3cc00-4650-4700-b30d-17b621fb8d28-socket-dir\") pod \"aws-ebs-csi-driver-node-b8d5j\" (UID: \"e5e3cc00-4650-4700-b30d-17b621fb8d28\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b8d5j"
Jul 06 22:23:54.697698 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.697620    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6a3050e9-b393-4767-a9c2-dafdf2a17925-host\") pod \"node-ca-9l9l5\" (UID: \"6a3050e9-b393-4767-a9c2-dafdf2a17925\") " pod="openshift-image-registry/node-ca-9l9l5"
Jul 06 22:23:54.698241 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.697725    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/55b0aff6-b51e-4948-8a6f-04eb6756ab6e-var-lib-kubelet\") pod \"tuned-8dl2h\" (UID: \"55b0aff6-b51e-4948-8a6f-04eb6756ab6e\") " pod="openshift-cluster-node-tuning-operator/tuned-8dl2h"
Jul 06 22:23:54.698241 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.697754    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/55b0aff6-b51e-4948-8a6f-04eb6756ab6e-host\") pod \"tuned-8dl2h\" (UID: \"55b0aff6-b51e-4948-8a6f-04eb6756ab6e\") " pod="openshift-cluster-node-tuning-operator/tuned-8dl2h"
Jul 06 22:23:54.698241 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.697790    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/79685e2c-e9e0-4ebe-93e6-12dd50c3c75c-host-var-lib-kubelet\") pod \"multus-vs62k\" (UID: \"79685e2c-e9e0-4ebe-93e6-12dd50c3c75c\") " pod="openshift-multus/multus-vs62k"
Jul 06 22:23:54.698241 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.697830    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9902a376-c450-4867-9c7c-d9e5300bce11-cnibin\") pod \"multus-additional-cni-plugins-rzszm\" (UID: \"9902a376-c450-4867-9c7c-d9e5300bce11\") " pod="openshift-multus/multus-additional-cni-plugins-rzszm"
Jul 06 22:23:54.698241 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.697848    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/69b0b78e-e934-433b-bfdb-5fd6d8ba4a29-metrics-certs\") pod \"network-metrics-daemon-bbjgn\" (UID: \"69b0b78e-e934-433b-bfdb-5fd6d8ba4a29\") " pod="openshift-multus/network-metrics-daemon-bbjgn"
Jul 06 22:23:54.698241 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.697887    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/79685e2c-e9e0-4ebe-93e6-12dd50c3c75c-host-var-lib-kubelet\") pod \"multus-vs62k\" (UID: \"79685e2c-e9e0-4ebe-93e6-12dd50c3c75c\") " pod="openshift-multus/multus-vs62k"
Jul 06 22:23:54.698241 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.697914    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/55b0aff6-b51e-4948-8a6f-04eb6756ab6e-lib-modules\") pod \"tuned-8dl2h\" (UID: \"55b0aff6-b51e-4948-8a6f-04eb6756ab6e\") " pod="openshift-cluster-node-tuning-operator/tuned-8dl2h"
Jul 06 22:23:54.698241 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.697929    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/79685e2c-e9e0-4ebe-93e6-12dd50c3c75c-cni-binary-copy\") pod \"multus-vs62k\" (UID: \"79685e2c-e9e0-4ebe-93e6-12dd50c3c75c\") " pod="openshift-multus/multus-vs62k"
Jul 06 22:23:54.698241 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.697943    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/03309f33-ff47-447d-956f-c89bdd4f8054-var-lib-openvswitch\") pod \"ovnkube-node-t6jpk\" (UID: \"03309f33-ff47-447d-956f-c89bdd4f8054\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6jpk"
Jul 06 22:23:54.698241 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.697979    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/03309f33-ff47-447d-956f-c89bdd4f8054-etc-openvswitch\") pod \"ovnkube-node-t6jpk\" (UID: \"03309f33-ff47-447d-956f-c89bdd4f8054\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6jpk"
Jul 06 22:23:54.698241 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.698003    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/03309f33-ff47-447d-956f-c89bdd4f8054-run-ovn\") pod \"ovnkube-node-t6jpk\" (UID: \"03309f33-ff47-447d-956f-c89bdd4f8054\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6jpk"
Jul 06 22:23:54.698241 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.698025    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pcrdh\" (UniqueName: \"kubernetes.io/projected/03309f33-ff47-447d-956f-c89bdd4f8054-kube-api-access-pcrdh\") pod \"ovnkube-node-t6jpk\" (UID: \"03309f33-ff47-447d-956f-c89bdd4f8054\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6jpk"
Jul 06 22:23:54.698241 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.698045    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e5e3cc00-4650-4700-b30d-17b621fb8d28-etc-selinux\") pod \"aws-ebs-csi-driver-node-b8d5j\" (UID: \"e5e3cc00-4650-4700-b30d-17b621fb8d28\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b8d5j"
Jul 06 22:23:54.698241 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.698062    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9902a376-c450-4867-9c7c-d9e5300bce11-os-release\") pod \"multus-additional-cni-plugins-rzszm\" (UID: \"9902a376-c450-4867-9c7c-d9e5300bce11\") " pod="openshift-multus/multus-additional-cni-plugins-rzszm"
Jul 06 22:23:54.698241 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.698083    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qbfbg\" (UniqueName: \"kubernetes.io/projected/69b0b78e-e934-433b-bfdb-5fd6d8ba4a29-kube-api-access-qbfbg\") pod \"network-metrics-daemon-bbjgn\" (UID: \"69b0b78e-e934-433b-bfdb-5fd6d8ba4a29\") " pod="openshift-multus/network-metrics-daemon-bbjgn"
Jul 06 22:23:54.698241 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.698118    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/55b0aff6-b51e-4948-8a6f-04eb6756ab6e-etc-modprobe-d\") pod \"tuned-8dl2h\" (UID: \"55b0aff6-b51e-4948-8a6f-04eb6756ab6e\") " pod="openshift-cluster-node-tuning-operator/tuned-8dl2h"
Jul 06 22:23:54.698241 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.698136    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/79685e2c-e9e0-4ebe-93e6-12dd50c3c75c-multus-cni-dir\") pod \"multus-vs62k\" (UID: \"79685e2c-e9e0-4ebe-93e6-12dd50c3c75c\") " pod="openshift-multus/multus-vs62k"
Jul 06 22:23:54.698947 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.698151    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/79685e2c-e9e0-4ebe-93e6-12dd50c3c75c-host-run-k8s-cni-cncf-io\") pod \"multus-vs62k\" (UID: \"79685e2c-e9e0-4ebe-93e6-12dd50c3c75c\") " pod="openshift-multus/multus-vs62k"
Jul 06 22:23:54.698947 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.698188    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e5e3cc00-4650-4700-b30d-17b621fb8d28-registration-dir\") pod \"aws-ebs-csi-driver-node-b8d5j\" (UID: \"e5e3cc00-4650-4700-b30d-17b621fb8d28\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b8d5j"
Jul 06 22:23:54.698947 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.698205    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e5e3cc00-4650-4700-b30d-17b621fb8d28-device-dir\") pod \"aws-ebs-csi-driver-node-b8d5j\" (UID: \"e5e3cc00-4650-4700-b30d-17b621fb8d28\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b8d5j"
Jul 06 22:23:54.698947 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.698221    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/55b0aff6-b51e-4948-8a6f-04eb6756ab6e-etc-kubernetes\") pod \"tuned-8dl2h\" (UID: \"55b0aff6-b51e-4948-8a6f-04eb6756ab6e\") " pod="openshift-cluster-node-tuning-operator/tuned-8dl2h"
Jul 06 22:23:54.698947 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.698234    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/55b0aff6-b51e-4948-8a6f-04eb6756ab6e-etc-tuned\") pod \"tuned-8dl2h\" (UID: \"55b0aff6-b51e-4948-8a6f-04eb6756ab6e\") " pod="openshift-cluster-node-tuning-operator/tuned-8dl2h"
Jul 06 22:23:54.698947 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.698249    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/79685e2c-e9e0-4ebe-93e6-12dd50c3c75c-cnibin\") pod \"multus-vs62k\" (UID: \"79685e2c-e9e0-4ebe-93e6-12dd50c3c75c\") " pod="openshift-multus/multus-vs62k"
Jul 06 22:23:54.698947 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.698264    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/79685e2c-e9e0-4ebe-93e6-12dd50c3c75c-host-var-lib-cni-bin\") pod \"multus-vs62k\" (UID: \"79685e2c-e9e0-4ebe-93e6-12dd50c3c75c\") " pod="openshift-multus/multus-vs62k"
Jul 06 22:23:54.698947 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.698294    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/03309f33-ff47-447d-956f-c89bdd4f8054-log-socket\") pod \"ovnkube-node-t6jpk\" (UID: \"03309f33-ff47-447d-956f-c89bdd4f8054\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6jpk"
Jul 06 22:23:54.698947 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.698309    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e5e3cc00-4650-4700-b30d-17b621fb8d28-kubelet-dir\") pod \"aws-ebs-csi-driver-node-b8d5j\" (UID: \"e5e3cc00-4650-4700-b30d-17b621fb8d28\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b8d5j"
Jul 06 22:23:54.698947 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.698312    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/79685e2c-e9e0-4ebe-93e6-12dd50c3c75c-multus-cni-dir\") pod \"multus-vs62k\" (UID: \"79685e2c-e9e0-4ebe-93e6-12dd50c3c75c\") " pod="openshift-multus/multus-vs62k"
Jul 06 22:23:54.698947 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.698324    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/809882c8-1856-4dc4-ac3f-710954ae19c7-host-slash\") pod \"iptables-alerter-slmxp\" (UID: \"809882c8-1856-4dc4-ac3f-710954ae19c7\") " pod="openshift-network-operator/iptables-alerter-slmxp"
Jul 06 22:23:54.698947 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.698344    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/79685e2c-e9e0-4ebe-93e6-12dd50c3c75c-os-release\") pod \"multus-vs62k\" (UID: \"79685e2c-e9e0-4ebe-93e6-12dd50c3c75c\") " pod="openshift-multus/multus-vs62k"
Jul 06 22:23:54.698947 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.698358    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/03309f33-ff47-447d-956f-c89bdd4f8054-host-run-ovn-kubernetes\") pod \"ovnkube-node-t6jpk\" (UID: \"03309f33-ff47-447d-956f-c89bdd4f8054\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6jpk"
Jul 06 22:23:54.698947 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.698372    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e5e3cc00-4650-4700-b30d-17b621fb8d28-sys-fs\") pod \"aws-ebs-csi-driver-node-b8d5j\" (UID: \"e5e3cc00-4650-4700-b30d-17b621fb8d28\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b8d5j"
Jul 06 22:23:54.698947 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.698373    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/79685e2c-e9e0-4ebe-93e6-12dd50c3c75c-host-run-k8s-cni-cncf-io\") pod \"multus-vs62k\" (UID: \"79685e2c-e9e0-4ebe-93e6-12dd50c3c75c\") " pod="openshift-multus/multus-vs62k"
Jul 06 22:23:54.698947 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.698387    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/55b0aff6-b51e-4948-8a6f-04eb6756ab6e-etc-systemd\") pod \"tuned-8dl2h\" (UID: \"55b0aff6-b51e-4948-8a6f-04eb6756ab6e\") " pod="openshift-cluster-node-tuning-operator/tuned-8dl2h"
Jul 06 22:23:54.698947 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.698438    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/79685e2c-e9e0-4ebe-93e6-12dd50c3c75c-cnibin\") pod \"multus-vs62k\" (UID: \"79685e2c-e9e0-4ebe-93e6-12dd50c3c75c\") " pod="openshift-multus/multus-vs62k"
Jul 06 22:23:54.699593 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.698465    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/55b0aff6-b51e-4948-8a6f-04eb6756ab6e-sys\") pod \"tuned-8dl2h\" (UID: \"55b0aff6-b51e-4948-8a6f-04eb6756ab6e\") " pod="openshift-cluster-node-tuning-operator/tuned-8dl2h"
Jul 06 22:23:54.699593 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.698482    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/03309f33-ff47-447d-956f-c89bdd4f8054-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-t6jpk\" (UID: \"03309f33-ff47-447d-956f-c89bdd4f8054\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6jpk"
Jul 06 22:23:54.699593 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.698486    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/79685e2c-e9e0-4ebe-93e6-12dd50c3c75c-host-var-lib-cni-bin\") pod \"multus-vs62k\" (UID: \"79685e2c-e9e0-4ebe-93e6-12dd50c3c75c\") " pod="openshift-multus/multus-vs62k"
Jul 06 22:23:54.699593 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.698509    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/03309f33-ff47-447d-956f-c89bdd4f8054-ovn-node-metrics-cert\") pod \"ovnkube-node-t6jpk\" (UID: \"03309f33-ff47-447d-956f-c89bdd4f8054\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6jpk"
Jul 06 22:23:54.699593 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.698533    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9902a376-c450-4867-9c7c-d9e5300bce11-tuning-conf-dir\") pod \"multus-additional-cni-plugins-rzszm\" (UID: \"9902a376-c450-4867-9c7c-d9e5300bce11\") " pod="openshift-multus/multus-additional-cni-plugins-rzszm"
Jul 06 22:23:54.699593 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.698552    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dgzgf\" (UniqueName: \"kubernetes.io/projected/55b0aff6-b51e-4948-8a6f-04eb6756ab6e-kube-api-access-dgzgf\") pod \"tuned-8dl2h\" (UID: \"55b0aff6-b51e-4948-8a6f-04eb6756ab6e\") " pod="openshift-cluster-node-tuning-operator/tuned-8dl2h"
Jul 06 22:23:54.699593 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.698577    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fzmgw\" (UniqueName: \"kubernetes.io/projected/9c27bdd9-eca8-4ef2-a260-56bb21de6b1e-kube-api-access-fzmgw\") pod \"node-resolver-tgnb8\" (UID: \"9c27bdd9-eca8-4ef2-a260-56bb21de6b1e\") " pod="openshift-dns/node-resolver-tgnb8"
Jul 06 22:23:54.699593 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.698592    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/03309f33-ff47-447d-956f-c89bdd4f8054-run-openvswitch\") pod \"ovnkube-node-t6jpk\" (UID: \"03309f33-ff47-447d-956f-c89bdd4f8054\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6jpk"
Jul 06 22:23:54.699593 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.698576    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/79685e2c-e9e0-4ebe-93e6-12dd50c3c75c-os-release\") pod \"multus-vs62k\" (UID: \"79685e2c-e9e0-4ebe-93e6-12dd50c3c75c\") " pod="openshift-multus/multus-vs62k"
Jul 06 22:23:54.699593 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.698619    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mntz2\" (UniqueName: \"kubernetes.io/projected/e5e3cc00-4650-4700-b30d-17b621fb8d28-kube-api-access-mntz2\") pod \"aws-ebs-csi-driver-node-b8d5j\" (UID: \"e5e3cc00-4650-4700-b30d-17b621fb8d28\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b8d5j"
Jul 06 22:23:54.699593 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.698733    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/6a3050e9-b393-4767-a9c2-dafdf2a17925-serviceca\") pod \"node-ca-9l9l5\" (UID: \"6a3050e9-b393-4767-a9c2-dafdf2a17925\") " pod="openshift-image-registry/node-ca-9l9l5"
Jul 06 22:23:54.699593 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.698759    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9902a376-c450-4867-9c7c-d9e5300bce11-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-rzszm\" (UID: \"9902a376-c450-4867-9c7c-d9e5300bce11\") " pod="openshift-multus/multus-additional-cni-plugins-rzszm"
Jul 06 22:23:54.699593 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.698788    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wtqwf\" (UniqueName: \"kubernetes.io/projected/9902a376-c450-4867-9c7c-d9e5300bce11-kube-api-access-wtqwf\") pod \"multus-additional-cni-plugins-rzszm\" (UID: \"9902a376-c450-4867-9c7c-d9e5300bce11\") " pod="openshift-multus/multus-additional-cni-plugins-rzszm"
Jul 06 22:23:54.699593 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.698807    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/809882c8-1856-4dc4-ac3f-710954ae19c7-iptables-alerter-script\") pod \"iptables-alerter-slmxp\" (UID: \"809882c8-1856-4dc4-ac3f-710954ae19c7\") " pod="openshift-network-operator/iptables-alerter-slmxp"
Jul 06 22:23:54.699593 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.698833    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/55b0aff6-b51e-4948-8a6f-04eb6756ab6e-etc-sysconfig\") pod \"tuned-8dl2h\" (UID: \"55b0aff6-b51e-4948-8a6f-04eb6756ab6e\") " pod="openshift-cluster-node-tuning-operator/tuned-8dl2h"
Jul 06 22:23:54.699593 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.698851    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/79685e2c-e9e0-4ebe-93e6-12dd50c3c75c-hostroot\") pod \"multus-vs62k\" (UID: \"79685e2c-e9e0-4ebe-93e6-12dd50c3c75c\") " pod="openshift-multus/multus-vs62k"
Jul 06 22:23:54.700048 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.698876    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/aad76575-952c-4b85-b64a-2523529c1cf4-dbus\") pod \"global-pull-secret-syncer-mc48r\" (UID: \"aad76575-952c-4b85-b64a-2523529c1cf4\") " pod="kube-system/global-pull-secret-syncer-mc48r"
Jul 06 22:23:54.700048 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.698906    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/79685e2c-e9e0-4ebe-93e6-12dd50c3c75c-hostroot\") pod \"multus-vs62k\" (UID: \"79685e2c-e9e0-4ebe-93e6-12dd50c3c75c\") " pod="openshift-multus/multus-vs62k"
Jul 06 22:23:54.700048 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.698936    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/79685e2c-e9e0-4ebe-93e6-12dd50c3c75c-host-var-lib-cni-multus\") pod \"multus-vs62k\" (UID: \"79685e2c-e9e0-4ebe-93e6-12dd50c3c75c\") " pod="openshift-multus/multus-vs62k"
Jul 06 22:23:54.700048 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.698952    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/79685e2c-e9e0-4ebe-93e6-12dd50c3c75c-multus-conf-dir\") pod \"multus-vs62k\" (UID: \"79685e2c-e9e0-4ebe-93e6-12dd50c3c75c\") " pod="openshift-multus/multus-vs62k"
Jul 06 22:23:54.700048 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.698968    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gvmwj\" (UniqueName: \"kubernetes.io/projected/da82c66a-1f8a-4315-8f45-8b449e2e46b9-kube-api-access-gvmwj\") pod \"network-check-target-6qmpm\" (UID: \"da82c66a-1f8a-4315-8f45-8b449e2e46b9\") " pod="openshift-network-diagnostics/network-check-target-6qmpm"
Jul 06 22:23:54.700048 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.698971    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/79685e2c-e9e0-4ebe-93e6-12dd50c3c75c-host-var-lib-cni-multus\") pod \"multus-vs62k\" (UID: \"79685e2c-e9e0-4ebe-93e6-12dd50c3c75c\") " pod="openshift-multus/multus-vs62k"
Jul 06 22:23:54.700048 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.698985    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/aad76575-952c-4b85-b64a-2523529c1cf4-original-pull-secret\") pod \"global-pull-secret-syncer-mc48r\" (UID: \"aad76575-952c-4b85-b64a-2523529c1cf4\") " pod="kube-system/global-pull-secret-syncer-mc48r"
Jul 06 22:23:54.700048 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.699003    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/686a5812-1550-4164-8e13-080cf1953bdb-agent-certs\") pod \"konnectivity-agent-z88v4\" (UID: \"686a5812-1550-4164-8e13-080cf1953bdb\") " pod="kube-system/konnectivity-agent-z88v4"
Jul 06 22:23:54.700048 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.699008    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/79685e2c-e9e0-4ebe-93e6-12dd50c3c75c-multus-conf-dir\") pod \"multus-vs62k\" (UID: \"79685e2c-e9e0-4ebe-93e6-12dd50c3c75c\") " pod="openshift-multus/multus-vs62k"
Jul 06 22:23:54.700048 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.699024    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9902a376-c450-4867-9c7c-d9e5300bce11-system-cni-dir\") pod \"multus-additional-cni-plugins-rzszm\" (UID: \"9902a376-c450-4867-9c7c-d9e5300bce11\") " pod="openshift-multus/multus-additional-cni-plugins-rzszm"
Jul 06 22:23:54.700048 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.699103    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/55b0aff6-b51e-4948-8a6f-04eb6756ab6e-run\") pod \"tuned-8dl2h\" (UID: \"55b0aff6-b51e-4948-8a6f-04eb6756ab6e\") " pod="openshift-cluster-node-tuning-operator/tuned-8dl2h"
Jul 06 22:23:54.700048 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.699120    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/9c27bdd9-eca8-4ef2-a260-56bb21de6b1e-hosts-file\") pod \"node-resolver-tgnb8\" (UID: \"9c27bdd9-eca8-4ef2-a260-56bb21de6b1e\") " pod="openshift-dns/node-resolver-tgnb8"
Jul 06 22:23:54.700048 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.699140    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/03309f33-ff47-447d-956f-c89bdd4f8054-systemd-units\") pod \"ovnkube-node-t6jpk\" (UID: \"03309f33-ff47-447d-956f-c89bdd4f8054\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6jpk"
Jul 06 22:23:54.700048 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.699201    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/9c27bdd9-eca8-4ef2-a260-56bb21de6b1e-hosts-file\") pod \"node-resolver-tgnb8\" (UID: \"9c27bdd9-eca8-4ef2-a260-56bb21de6b1e\") " pod="openshift-dns/node-resolver-tgnb8"
Jul 06 22:23:54.700048 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.699205    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/03309f33-ff47-447d-956f-c89bdd4f8054-run-systemd\") pod \"ovnkube-node-t6jpk\" (UID: \"03309f33-ff47-447d-956f-c89bdd4f8054\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6jpk"
Jul 06 22:23:54.700048 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.699251    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/03309f33-ff47-447d-956f-c89bdd4f8054-node-log\") pod \"ovnkube-node-t6jpk\" (UID: \"03309f33-ff47-447d-956f-c89bdd4f8054\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6jpk"
Jul 06 22:23:54.700048 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.699285    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/79685e2c-e9e0-4ebe-93e6-12dd50c3c75c-multus-daemon-config\") pod \"multus-vs62k\" (UID: \"79685e2c-e9e0-4ebe-93e6-12dd50c3c75c\") " pod="openshift-multus/multus-vs62k"
Jul 06 22:23:54.700531 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.699353    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/03309f33-ff47-447d-956f-c89bdd4f8054-env-overrides\") pod \"ovnkube-node-t6jpk\" (UID: \"03309f33-ff47-447d-956f-c89bdd4f8054\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6jpk"
Jul 06 22:23:54.700531 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.699395    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9902a376-c450-4867-9c7c-d9e5300bce11-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-rzszm\" (UID: \"9902a376-c450-4867-9c7c-d9e5300bce11\") " pod="openshift-multus/multus-additional-cni-plugins-rzszm"
Jul 06 22:23:54.700531 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.699436    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/79685e2c-e9e0-4ebe-93e6-12dd50c3c75c-system-cni-dir\") pod \"multus-vs62k\" (UID: \"79685e2c-e9e0-4ebe-93e6-12dd50c3c75c\") " pod="openshift-multus/multus-vs62k"
Jul 06 22:23:54.700531 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.699534    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/79685e2c-e9e0-4ebe-93e6-12dd50c3c75c-system-cni-dir\") pod \"multus-vs62k\" (UID: \"79685e2c-e9e0-4ebe-93e6-12dd50c3c75c\") " pod="openshift-multus/multus-vs62k"
Jul 06 22:23:54.700531 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.699636    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/79685e2c-e9e0-4ebe-93e6-12dd50c3c75c-multus-daemon-config\") pod \"multus-vs62k\" (UID: \"79685e2c-e9e0-4ebe-93e6-12dd50c3c75c\") " pod="openshift-multus/multus-vs62k"
Jul 06 22:23:54.702231 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.702214    2563 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 06 22:23:54.703848 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:23:54.703780    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 06 22:23:54.703848 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:23:54.703804    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 06 22:23:54.703848 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:23:54.703818    2563 projected.go:196] Error preparing data for projected volume kube-api-access-gvmwj for pod openshift-network-diagnostics/network-check-target-6qmpm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 22:23:54.703848 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:23:54.703895    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/da82c66a-1f8a-4315-8f45-8b449e2e46b9-kube-api-access-gvmwj podName:da82c66a-1f8a-4315-8f45-8b449e2e46b9 nodeName:}" failed. No retries permitted until 2026-07-06 22:23:55.203874576 +0000 UTC m=+2.013921952 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-gvmwj" (UniqueName: "kubernetes.io/projected/da82c66a-1f8a-4315-8f45-8b449e2e46b9-kube-api-access-gvmwj") pod "network-check-target-6qmpm" (UID: "da82c66a-1f8a-4315-8f45-8b449e2e46b9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 22:23:54.705461 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.705438    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8vk4k\" (UniqueName: \"kubernetes.io/projected/79685e2c-e9e0-4ebe-93e6-12dd50c3c75c-kube-api-access-8vk4k\") pod \"multus-vs62k\" (UID: \"79685e2c-e9e0-4ebe-93e6-12dd50c3c75c\") " pod="openshift-multus/multus-vs62k"
Jul 06 22:23:54.705461 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.705449    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fzmgw\" (UniqueName: \"kubernetes.io/projected/9c27bdd9-eca8-4ef2-a260-56bb21de6b1e-kube-api-access-fzmgw\") pod \"node-resolver-tgnb8\" (UID: \"9c27bdd9-eca8-4ef2-a260-56bb21de6b1e\") " pod="openshift-dns/node-resolver-tgnb8"
Jul 06 22:23:54.750572 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:54.750526    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80e55e8f9c22a5bcba25f5b76deba1dd.slice/crio-aab95d1c2871b7f6f6749f2b503fa8e0f4742212d8b944ad45365682aa610a87 WatchSource:0}: Error finding container aab95d1c2871b7f6f6749f2b503fa8e0f4742212d8b944ad45365682aa610a87: Status 404 returned error can't find the container with id aab95d1c2871b7f6f6749f2b503fa8e0f4742212d8b944ad45365682aa610a87
Jul 06 22:23:54.751228 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:54.750860    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7e777c6384fb2d7e87381cc5d382585.slice/crio-2471430135b24cea418b887512cb4a77fc3559d77c3f8fc2ce56baf7592b933b WatchSource:0}: Error finding container 2471430135b24cea418b887512cb4a77fc3559d77c3f8fc2ce56baf7592b933b: Status 404 returned error can't find the container with id 2471430135b24cea418b887512cb4a77fc3559d77c3f8fc2ce56baf7592b933b
Jul 06 22:23:54.756036 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.756009    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 06 22:23:54.799949 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.799924    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/03309f33-ff47-447d-956f-c89bdd4f8054-log-socket\") pod \"ovnkube-node-t6jpk\" (UID: \"03309f33-ff47-447d-956f-c89bdd4f8054\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6jpk"
Jul 06 22:23:54.799949 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.799952    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e5e3cc00-4650-4700-b30d-17b621fb8d28-kubelet-dir\") pod \"aws-ebs-csi-driver-node-b8d5j\" (UID: \"e5e3cc00-4650-4700-b30d-17b621fb8d28\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b8d5j"
Jul 06 22:23:54.800214 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.799969    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/809882c8-1856-4dc4-ac3f-710954ae19c7-host-slash\") pod \"iptables-alerter-slmxp\" (UID: \"809882c8-1856-4dc4-ac3f-710954ae19c7\") " pod="openshift-network-operator/iptables-alerter-slmxp"
Jul 06 22:23:54.800214 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.799987    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/03309f33-ff47-447d-956f-c89bdd4f8054-host-run-ovn-kubernetes\") pod \"ovnkube-node-t6jpk\" (UID: \"03309f33-ff47-447d-956f-c89bdd4f8054\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6jpk"
Jul 06 22:23:54.800214 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.800003    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e5e3cc00-4650-4700-b30d-17b621fb8d28-sys-fs\") pod \"aws-ebs-csi-driver-node-b8d5j\" (UID: \"e5e3cc00-4650-4700-b30d-17b621fb8d28\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b8d5j"
Jul 06 22:23:54.800214 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.800026    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/55b0aff6-b51e-4948-8a6f-04eb6756ab6e-etc-systemd\") pod \"tuned-8dl2h\" (UID: \"55b0aff6-b51e-4948-8a6f-04eb6756ab6e\") " pod="openshift-cluster-node-tuning-operator/tuned-8dl2h"
Jul 06 22:23:54.800214 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.800055    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/03309f33-ff47-447d-956f-c89bdd4f8054-log-socket\") pod \"ovnkube-node-t6jpk\" (UID: \"03309f33-ff47-447d-956f-c89bdd4f8054\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6jpk"
Jul 06 22:23:54.800214 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.800065    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/03309f33-ff47-447d-956f-c89bdd4f8054-host-run-ovn-kubernetes\") pod \"ovnkube-node-t6jpk\" (UID: \"03309f33-ff47-447d-956f-c89bdd4f8054\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6jpk"
Jul 06 22:23:54.800214 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.800075    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/55b0aff6-b51e-4948-8a6f-04eb6756ab6e-etc-systemd\") pod \"tuned-8dl2h\" (UID: \"55b0aff6-b51e-4948-8a6f-04eb6756ab6e\") " pod="openshift-cluster-node-tuning-operator/tuned-8dl2h"
Jul 06 22:23:54.800214 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.800056    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/809882c8-1856-4dc4-ac3f-710954ae19c7-host-slash\") pod \"iptables-alerter-slmxp\" (UID: \"809882c8-1856-4dc4-ac3f-710954ae19c7\") " pod="openshift-network-operator/iptables-alerter-slmxp"
Jul 06 22:23:54.800214 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.800065    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e5e3cc00-4650-4700-b30d-17b621fb8d28-kubelet-dir\") pod \"aws-ebs-csi-driver-node-b8d5j\" (UID: \"e5e3cc00-4650-4700-b30d-17b621fb8d28\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b8d5j"
Jul 06 22:23:54.800214 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.800093    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/55b0aff6-b51e-4948-8a6f-04eb6756ab6e-sys\") pod \"tuned-8dl2h\" (UID: \"55b0aff6-b51e-4948-8a6f-04eb6756ab6e\") " pod="openshift-cluster-node-tuning-operator/tuned-8dl2h"
Jul 06 22:23:54.800214 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.800103    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e5e3cc00-4650-4700-b30d-17b621fb8d28-sys-fs\") pod \"aws-ebs-csi-driver-node-b8d5j\" (UID: \"e5e3cc00-4650-4700-b30d-17b621fb8d28\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b8d5j"
Jul 06 22:23:54.800214 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.800123    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/03309f33-ff47-447d-956f-c89bdd4f8054-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-t6jpk\" (UID: \"03309f33-ff47-447d-956f-c89bdd4f8054\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6jpk"
Jul 06 22:23:54.800214 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.800129    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/55b0aff6-b51e-4948-8a6f-04eb6756ab6e-sys\") pod \"tuned-8dl2h\" (UID: \"55b0aff6-b51e-4948-8a6f-04eb6756ab6e\") " pod="openshift-cluster-node-tuning-operator/tuned-8dl2h"
Jul 06 22:23:54.800214 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.800154    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/03309f33-ff47-447d-956f-c89bdd4f8054-ovn-node-metrics-cert\") pod \"ovnkube-node-t6jpk\" (UID: \"03309f33-ff47-447d-956f-c89bdd4f8054\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6jpk"
Jul 06 22:23:54.800214 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.800186    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/03309f33-ff47-447d-956f-c89bdd4f8054-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-t6jpk\" (UID: \"03309f33-ff47-447d-956f-c89bdd4f8054\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6jpk"
Jul 06 22:23:54.800214 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.800196    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9902a376-c450-4867-9c7c-d9e5300bce11-tuning-conf-dir\") pod \"multus-additional-cni-plugins-rzszm\" (UID: \"9902a376-c450-4867-9c7c-d9e5300bce11\") " pod="openshift-multus/multus-additional-cni-plugins-rzszm"
Jul 06 22:23:54.800955 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.800230    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dgzgf\" (UniqueName: \"kubernetes.io/projected/55b0aff6-b51e-4948-8a6f-04eb6756ab6e-kube-api-access-dgzgf\") pod \"tuned-8dl2h\" (UID: \"55b0aff6-b51e-4948-8a6f-04eb6756ab6e\") " pod="openshift-cluster-node-tuning-operator/tuned-8dl2h"
Jul 06 22:23:54.800955 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.800255    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/03309f33-ff47-447d-956f-c89bdd4f8054-run-openvswitch\") pod \"ovnkube-node-t6jpk\" (UID: \"03309f33-ff47-447d-956f-c89bdd4f8054\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6jpk"
Jul 06 22:23:54.800955 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.800275    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mntz2\" (UniqueName: \"kubernetes.io/projected/e5e3cc00-4650-4700-b30d-17b621fb8d28-kube-api-access-mntz2\") pod \"aws-ebs-csi-driver-node-b8d5j\" (UID: \"e5e3cc00-4650-4700-b30d-17b621fb8d28\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b8d5j"
Jul 06 22:23:54.800955 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.800295    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/6a3050e9-b393-4767-a9c2-dafdf2a17925-serviceca\") pod \"node-ca-9l9l5\" (UID: \"6a3050e9-b393-4767-a9c2-dafdf2a17925\") " pod="openshift-image-registry/node-ca-9l9l5"
Jul 06 22:23:54.800955 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.800320    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9902a376-c450-4867-9c7c-d9e5300bce11-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-rzszm\" (UID: \"9902a376-c450-4867-9c7c-d9e5300bce11\") " pod="openshift-multus/multus-additional-cni-plugins-rzszm"
Jul 06 22:23:54.800955 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.800330    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/03309f33-ff47-447d-956f-c89bdd4f8054-run-openvswitch\") pod \"ovnkube-node-t6jpk\" (UID: \"03309f33-ff47-447d-956f-c89bdd4f8054\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6jpk"
Jul 06 22:23:54.800955 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.800363    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9902a376-c450-4867-9c7c-d9e5300bce11-tuning-conf-dir\") pod \"multus-additional-cni-plugins-rzszm\" (UID: \"9902a376-c450-4867-9c7c-d9e5300bce11\") " pod="openshift-multus/multus-additional-cni-plugins-rzszm"
Jul 06 22:23:54.800955 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.800434    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wtqwf\" (UniqueName: \"kubernetes.io/projected/9902a376-c450-4867-9c7c-d9e5300bce11-kube-api-access-wtqwf\") pod \"multus-additional-cni-plugins-rzszm\" (UID: \"9902a376-c450-4867-9c7c-d9e5300bce11\") " pod="openshift-multus/multus-additional-cni-plugins-rzszm"
Jul 06 22:23:54.800955 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.800463    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/809882c8-1856-4dc4-ac3f-710954ae19c7-iptables-alerter-script\") pod \"iptables-alerter-slmxp\" (UID: \"809882c8-1856-4dc4-ac3f-710954ae19c7\") " pod="openshift-network-operator/iptables-alerter-slmxp"
Jul 06 22:23:54.800955 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.800512    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/55b0aff6-b51e-4948-8a6f-04eb6756ab6e-etc-sysconfig\") pod \"tuned-8dl2h\" (UID: \"55b0aff6-b51e-4948-8a6f-04eb6756ab6e\") " pod="openshift-cluster-node-tuning-operator/tuned-8dl2h"
Jul 06 22:23:54.800955 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.800538    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/aad76575-952c-4b85-b64a-2523529c1cf4-dbus\") pod \"global-pull-secret-syncer-mc48r\" (UID: \"aad76575-952c-4b85-b64a-2523529c1cf4\") " pod="kube-system/global-pull-secret-syncer-mc48r"
Jul 06 22:23:54.800955 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.800581    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/aad76575-952c-4b85-b64a-2523529c1cf4-original-pull-secret\") pod \"global-pull-secret-syncer-mc48r\" (UID: \"aad76575-952c-4b85-b64a-2523529c1cf4\") " pod="kube-system/global-pull-secret-syncer-mc48r"
Jul 06 22:23:54.800955 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:23:54.800667    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 06 22:23:54.800955 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.800690    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/686a5812-1550-4164-8e13-080cf1953bdb-agent-certs\") pod \"konnectivity-agent-z88v4\" (UID: \"686a5812-1550-4164-8e13-080cf1953bdb\") " pod="kube-system/konnectivity-agent-z88v4"
Jul 06 22:23:54.800955 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:23:54.800728    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aad76575-952c-4b85-b64a-2523529c1cf4-original-pull-secret podName:aad76575-952c-4b85-b64a-2523529c1cf4 nodeName:}" failed. No retries permitted until 2026-07-06 22:23:55.300712364 +0000 UTC m=+2.110759744 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/aad76575-952c-4b85-b64a-2523529c1cf4-original-pull-secret") pod "global-pull-secret-syncer-mc48r" (UID: "aad76575-952c-4b85-b64a-2523529c1cf4") : object "kube-system"/"original-pull-secret" not registered
Jul 06 22:23:54.800955 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.800756    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9902a376-c450-4867-9c7c-d9e5300bce11-system-cni-dir\") pod \"multus-additional-cni-plugins-rzszm\" (UID: \"9902a376-c450-4867-9c7c-d9e5300bce11\") " pod="openshift-multus/multus-additional-cni-plugins-rzszm"
Jul 06 22:23:54.800955 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.800781    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/55b0aff6-b51e-4948-8a6f-04eb6756ab6e-run\") pod \"tuned-8dl2h\" (UID: \"55b0aff6-b51e-4948-8a6f-04eb6756ab6e\") " pod="openshift-cluster-node-tuning-operator/tuned-8dl2h"
Jul 06 22:23:54.801842 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.800793    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/6a3050e9-b393-4767-a9c2-dafdf2a17925-serviceca\") pod \"node-ca-9l9l5\" (UID: \"6a3050e9-b393-4767-a9c2-dafdf2a17925\") " pod="openshift-image-registry/node-ca-9l9l5"
Jul 06 22:23:54.801842 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.800809    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/03309f33-ff47-447d-956f-c89bdd4f8054-systemd-units\") pod \"ovnkube-node-t6jpk\" (UID: \"03309f33-ff47-447d-956f-c89bdd4f8054\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6jpk"
Jul 06 22:23:54.801842 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.800754    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/aad76575-952c-4b85-b64a-2523529c1cf4-dbus\") pod \"global-pull-secret-syncer-mc48r\" (UID: \"aad76575-952c-4b85-b64a-2523529c1cf4\") " pod="kube-system/global-pull-secret-syncer-mc48r"
Jul 06 22:23:54.801842 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.800837    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/03309f33-ff47-447d-956f-c89bdd4f8054-run-systemd\") pod \"ovnkube-node-t6jpk\" (UID: \"03309f33-ff47-447d-956f-c89bdd4f8054\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6jpk"
Jul 06 22:23:54.801842 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.800861    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/03309f33-ff47-447d-956f-c89bdd4f8054-node-log\") pod \"ovnkube-node-t6jpk\" (UID: \"03309f33-ff47-447d-956f-c89bdd4f8054\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6jpk"
Jul 06 22:23:54.801842 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.800854    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9902a376-c450-4867-9c7c-d9e5300bce11-system-cni-dir\") pod \"multus-additional-cni-plugins-rzszm\" (UID: \"9902a376-c450-4867-9c7c-d9e5300bce11\") " pod="openshift-multus/multus-additional-cni-plugins-rzszm"
Jul 06 22:23:54.801842 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.800890    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/03309f33-ff47-447d-956f-c89bdd4f8054-env-overrides\") pod \"ovnkube-node-t6jpk\" (UID: \"03309f33-ff47-447d-956f-c89bdd4f8054\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6jpk"
Jul 06 22:23:54.801842 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.800861    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9902a376-c450-4867-9c7c-d9e5300bce11-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-rzszm\" (UID: \"9902a376-c450-4867-9c7c-d9e5300bce11\") " pod="openshift-multus/multus-additional-cni-plugins-rzszm"
Jul 06 22:23:54.801842 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.800917    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/03309f33-ff47-447d-956f-c89bdd4f8054-systemd-units\") pod \"ovnkube-node-t6jpk\" (UID: \"03309f33-ff47-447d-956f-c89bdd4f8054\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6jpk"
Jul 06 22:23:54.801842 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.800954    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/03309f33-ff47-447d-956f-c89bdd4f8054-run-systemd\") pod \"ovnkube-node-t6jpk\" (UID: \"03309f33-ff47-447d-956f-c89bdd4f8054\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6jpk"
Jul 06 22:23:54.801842 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.800962    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/55b0aff6-b51e-4948-8a6f-04eb6756ab6e-run\") pod \"tuned-8dl2h\" (UID: \"55b0aff6-b51e-4948-8a6f-04eb6756ab6e\") " pod="openshift-cluster-node-tuning-operator/tuned-8dl2h"
Jul 06 22:23:54.801842 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.800918    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/03309f33-ff47-447d-956f-c89bdd4f8054-node-log\") pod \"ovnkube-node-t6jpk\" (UID: \"03309f33-ff47-447d-956f-c89bdd4f8054\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6jpk"
Jul 06 22:23:54.801842 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.800646    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/55b0aff6-b51e-4948-8a6f-04eb6756ab6e-etc-sysconfig\") pod \"tuned-8dl2h\" (UID: \"55b0aff6-b51e-4948-8a6f-04eb6756ab6e\") " pod="openshift-cluster-node-tuning-operator/tuned-8dl2h"
Jul 06 22:23:54.801842 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.800916    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9902a376-c450-4867-9c7c-d9e5300bce11-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-rzszm\" (UID: \"9902a376-c450-4867-9c7c-d9e5300bce11\") " pod="openshift-multus/multus-additional-cni-plugins-rzszm"
Jul 06 22:23:54.801842 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.800979    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/809882c8-1856-4dc4-ac3f-710954ae19c7-iptables-alerter-script\") pod \"iptables-alerter-slmxp\" (UID: \"809882c8-1856-4dc4-ac3f-710954ae19c7\") " pod="openshift-network-operator/iptables-alerter-slmxp"
Jul 06 22:23:54.801842 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.801008    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/03309f33-ff47-447d-956f-c89bdd4f8054-host-cni-netd\") pod \"ovnkube-node-t6jpk\" (UID: \"03309f33-ff47-447d-956f-c89bdd4f8054\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6jpk"
Jul 06 22:23:54.801842 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.801033    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/03309f33-ff47-447d-956f-c89bdd4f8054-ovnkube-script-lib\") pod \"ovnkube-node-t6jpk\" (UID: \"03309f33-ff47-447d-956f-c89bdd4f8054\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6jpk"
Jul 06 22:23:54.802686 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.801066    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/aad76575-952c-4b85-b64a-2523529c1cf4-kubelet-config\") pod \"global-pull-secret-syncer-mc48r\" (UID: \"aad76575-952c-4b85-b64a-2523529c1cf4\") " pod="kube-system/global-pull-secret-syncer-mc48r"
Jul 06 22:23:54.802686 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.801090    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/686a5812-1550-4164-8e13-080cf1953bdb-konnectivity-ca\") pod \"konnectivity-agent-z88v4\" (UID: \"686a5812-1550-4164-8e13-080cf1953bdb\") " pod="kube-system/konnectivity-agent-z88v4"
Jul 06 22:23:54.802686 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.801109    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/03309f33-ff47-447d-956f-c89bdd4f8054-host-cni-netd\") pod \"ovnkube-node-t6jpk\" (UID: \"03309f33-ff47-447d-956f-c89bdd4f8054\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6jpk"
Jul 06 22:23:54.802686 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.801117    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/03309f33-ff47-447d-956f-c89bdd4f8054-host-slash\") pod \"ovnkube-node-t6jpk\" (UID: \"03309f33-ff47-447d-956f-c89bdd4f8054\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6jpk"
Jul 06 22:23:54.802686 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.801150    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/03309f33-ff47-447d-956f-c89bdd4f8054-host-slash\") pod \"ovnkube-node-t6jpk\" (UID: \"03309f33-ff47-447d-956f-c89bdd4f8054\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6jpk"
Jul 06 22:23:54.802686 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.801150    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ntg8q\" (UniqueName: \"kubernetes.io/projected/809882c8-1856-4dc4-ac3f-710954ae19c7-kube-api-access-ntg8q\") pod \"iptables-alerter-slmxp\" (UID: \"809882c8-1856-4dc4-ac3f-710954ae19c7\") " pod="openshift-network-operator/iptables-alerter-slmxp"
Jul 06 22:23:54.802686 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.801214    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/03309f33-ff47-447d-956f-c89bdd4f8054-host-kubelet\") pod \"ovnkube-node-t6jpk\" (UID: \"03309f33-ff47-447d-956f-c89bdd4f8054\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6jpk"
Jul 06 22:23:54.802686 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.801217    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/aad76575-952c-4b85-b64a-2523529c1cf4-kubelet-config\") pod \"global-pull-secret-syncer-mc48r\" (UID: \"aad76575-952c-4b85-b64a-2523529c1cf4\") " pod="kube-system/global-pull-secret-syncer-mc48r"
Jul 06 22:23:54.802686 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.801240    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/55b0aff6-b51e-4948-8a6f-04eb6756ab6e-etc-sysctl-conf\") pod \"tuned-8dl2h\" (UID: \"55b0aff6-b51e-4948-8a6f-04eb6756ab6e\") " pod="openshift-cluster-node-tuning-operator/tuned-8dl2h"
Jul 06 22:23:54.802686 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.801267    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/55b0aff6-b51e-4948-8a6f-04eb6756ab6e-tmp\") pod \"tuned-8dl2h\" (UID: \"55b0aff6-b51e-4948-8a6f-04eb6756ab6e\") " pod="openshift-cluster-node-tuning-operator/tuned-8dl2h"
Jul 06 22:23:54.802686 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.801290    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/03309f33-ff47-447d-956f-c89bdd4f8054-ovnkube-config\") pod \"ovnkube-node-t6jpk\" (UID: \"03309f33-ff47-447d-956f-c89bdd4f8054\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6jpk"
Jul 06 22:23:54.802686 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.801315    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dd78p\" (UniqueName: \"kubernetes.io/projected/6a3050e9-b393-4767-a9c2-dafdf2a17925-kube-api-access-dd78p\") pod \"node-ca-9l9l5\" (UID: \"6a3050e9-b393-4767-a9c2-dafdf2a17925\") " pod="openshift-image-registry/node-ca-9l9l5"
Jul 06 22:23:54.802686 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.801344    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9902a376-c450-4867-9c7c-d9e5300bce11-cni-binary-copy\") pod \"multus-additional-cni-plugins-rzszm\" (UID: \"9902a376-c450-4867-9c7c-d9e5300bce11\") " pod="openshift-multus/multus-additional-cni-plugins-rzszm"
Jul 06 22:23:54.802686 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.801368    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/55b0aff6-b51e-4948-8a6f-04eb6756ab6e-etc-sysctl-d\") pod \"tuned-8dl2h\" (UID: \"55b0aff6-b51e-4948-8a6f-04eb6756ab6e\") " pod="openshift-cluster-node-tuning-operator/tuned-8dl2h"
Jul 06 22:23:54.802686 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.801392    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/03309f33-ff47-447d-956f-c89bdd4f8054-host-run-netns\") pod \"ovnkube-node-t6jpk\" (UID: \"03309f33-ff47-447d-956f-c89bdd4f8054\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6jpk"
Jul 06 22:23:54.802686 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.801409    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9902a376-c450-4867-9c7c-d9e5300bce11-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-rzszm\" (UID: \"9902a376-c450-4867-9c7c-d9e5300bce11\") " pod="openshift-multus/multus-additional-cni-plugins-rzszm"
Jul 06 22:23:54.802686 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.801419    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/03309f33-ff47-447d-956f-c89bdd4f8054-host-cni-bin\") pod \"ovnkube-node-t6jpk\" (UID: \"03309f33-ff47-447d-956f-c89bdd4f8054\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6jpk"
Jul 06 22:23:54.803703 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.801410    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/03309f33-ff47-447d-956f-c89bdd4f8054-env-overrides\") pod \"ovnkube-node-t6jpk\" (UID: \"03309f33-ff47-447d-956f-c89bdd4f8054\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6jpk"
Jul 06 22:23:54.803703 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.801452    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e5e3cc00-4650-4700-b30d-17b621fb8d28-socket-dir\") pod \"aws-ebs-csi-driver-node-b8d5j\" (UID: \"e5e3cc00-4650-4700-b30d-17b621fb8d28\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b8d5j"
Jul 06 22:23:54.803703 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.801463    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/03309f33-ff47-447d-956f-c89bdd4f8054-host-cni-bin\") pod \"ovnkube-node-t6jpk\" (UID: \"03309f33-ff47-447d-956f-c89bdd4f8054\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6jpk"
Jul 06 22:23:54.803703 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.801500    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6a3050e9-b393-4767-a9c2-dafdf2a17925-host\") pod \"node-ca-9l9l5\" (UID: \"6a3050e9-b393-4767-a9c2-dafdf2a17925\") " pod="openshift-image-registry/node-ca-9l9l5"
Jul 06 22:23:54.803703 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.801514    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/03309f33-ff47-447d-956f-c89bdd4f8054-host-kubelet\") pod \"ovnkube-node-t6jpk\" (UID: \"03309f33-ff47-447d-956f-c89bdd4f8054\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6jpk"
Jul 06 22:23:54.803703 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.801525    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/55b0aff6-b51e-4948-8a6f-04eb6756ab6e-var-lib-kubelet\") pod \"tuned-8dl2h\" (UID: \"55b0aff6-b51e-4948-8a6f-04eb6756ab6e\") " pod="openshift-cluster-node-tuning-operator/tuned-8dl2h"
Jul 06 22:23:54.803703 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.801554    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/55b0aff6-b51e-4948-8a6f-04eb6756ab6e-host\") pod \"tuned-8dl2h\" (UID: \"55b0aff6-b51e-4948-8a6f-04eb6756ab6e\") " pod="openshift-cluster-node-tuning-operator/tuned-8dl2h"
Jul 06 22:23:54.803703 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.801576    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9902a376-c450-4867-9c7c-d9e5300bce11-cnibin\") pod \"multus-additional-cni-plugins-rzszm\" (UID: \"9902a376-c450-4867-9c7c-d9e5300bce11\") " pod="openshift-multus/multus-additional-cni-plugins-rzszm"
Jul 06 22:23:54.803703 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.801599    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/69b0b78e-e934-433b-bfdb-5fd6d8ba4a29-metrics-certs\") pod \"network-metrics-daemon-bbjgn\" (UID: \"69b0b78e-e934-433b-bfdb-5fd6d8ba4a29\") " pod="openshift-multus/network-metrics-daemon-bbjgn"
Jul 06 22:23:54.803703 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.801624    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/55b0aff6-b51e-4948-8a6f-04eb6756ab6e-etc-sysctl-conf\") pod \"tuned-8dl2h\" (UID: \"55b0aff6-b51e-4948-8a6f-04eb6756ab6e\") " pod="openshift-cluster-node-tuning-operator/tuned-8dl2h"
Jul 06 22:23:54.803703 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.801663    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6a3050e9-b393-4767-a9c2-dafdf2a17925-host\") pod \"node-ca-9l9l5\" (UID: \"6a3050e9-b393-4767-a9c2-dafdf2a17925\") " pod="openshift-image-registry/node-ca-9l9l5"
Jul 06 22:23:54.803703 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.801711    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/55b0aff6-b51e-4948-8a6f-04eb6756ab6e-var-lib-kubelet\") pod \"tuned-8dl2h\" (UID: \"55b0aff6-b51e-4948-8a6f-04eb6756ab6e\") " pod="openshift-cluster-node-tuning-operator/tuned-8dl2h"
Jul 06 22:23:54.803703 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.801729    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e5e3cc00-4650-4700-b30d-17b621fb8d28-socket-dir\") pod \"aws-ebs-csi-driver-node-b8d5j\" (UID: \"e5e3cc00-4650-4700-b30d-17b621fb8d28\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b8d5j"
Jul 06 22:23:54.803703 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.801736    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/686a5812-1550-4164-8e13-080cf1953bdb-konnectivity-ca\") pod \"konnectivity-agent-z88v4\" (UID: \"686a5812-1550-4164-8e13-080cf1953bdb\") " pod="kube-system/konnectivity-agent-z88v4"
Jul 06 22:23:54.803703 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.801628    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/55b0aff6-b51e-4948-8a6f-04eb6756ab6e-lib-modules\") pod \"tuned-8dl2h\" (UID: \"55b0aff6-b51e-4948-8a6f-04eb6756ab6e\") " pod="openshift-cluster-node-tuning-operator/tuned-8dl2h"
Jul 06 22:23:54.803703 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.801752    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/55b0aff6-b51e-4948-8a6f-04eb6756ab6e-host\") pod \"tuned-8dl2h\" (UID: \"55b0aff6-b51e-4948-8a6f-04eb6756ab6e\") " pod="openshift-cluster-node-tuning-operator/tuned-8dl2h"
Jul 06 22:23:54.803703 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.801780    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9902a376-c450-4867-9c7c-d9e5300bce11-cnibin\") pod \"multus-additional-cni-plugins-rzszm\" (UID: \"9902a376-c450-4867-9c7c-d9e5300bce11\") " pod="openshift-multus/multus-additional-cni-plugins-rzszm"
Jul 06 22:23:54.803703 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.801803    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/03309f33-ff47-447d-956f-c89bdd4f8054-var-lib-openvswitch\") pod \"ovnkube-node-t6jpk\" (UID: \"03309f33-ff47-447d-956f-c89bdd4f8054\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6jpk"
Jul 06 22:23:54.805566 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.801830    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/03309f33-ff47-447d-956f-c89bdd4f8054-etc-openvswitch\") pod \"ovnkube-node-t6jpk\" (UID: \"03309f33-ff47-447d-956f-c89bdd4f8054\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6jpk"
Jul 06 22:23:54.805566 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:23:54.801852    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 22:23:54.805566 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.801872    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/55b0aff6-b51e-4948-8a6f-04eb6756ab6e-lib-modules\") pod \"tuned-8dl2h\" (UID: \"55b0aff6-b51e-4948-8a6f-04eb6756ab6e\") " pod="openshift-cluster-node-tuning-operator/tuned-8dl2h"
Jul 06 22:23:54.805566 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.801894    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/03309f33-ff47-447d-956f-c89bdd4f8054-run-ovn\") pod \"ovnkube-node-t6jpk\" (UID: \"03309f33-ff47-447d-956f-c89bdd4f8054\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6jpk"
Jul 06 22:23:54.805566 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.801854    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/03309f33-ff47-447d-956f-c89bdd4f8054-run-ovn\") pod \"ovnkube-node-t6jpk\" (UID: \"03309f33-ff47-447d-956f-c89bdd4f8054\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6jpk"
Jul 06 22:23:54.805566 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:23:54.801908    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/69b0b78e-e934-433b-bfdb-5fd6d8ba4a29-metrics-certs podName:69b0b78e-e934-433b-bfdb-5fd6d8ba4a29 nodeName:}" failed. No retries permitted until 2026-07-06 22:23:55.301892743 +0000 UTC m=+2.111940110 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/69b0b78e-e934-433b-bfdb-5fd6d8ba4a29-metrics-certs") pod "network-metrics-daemon-bbjgn" (UID: "69b0b78e-e934-433b-bfdb-5fd6d8ba4a29") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 22:23:54.805566 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.801945    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pcrdh\" (UniqueName: \"kubernetes.io/projected/03309f33-ff47-447d-956f-c89bdd4f8054-kube-api-access-pcrdh\") pod \"ovnkube-node-t6jpk\" (UID: \"03309f33-ff47-447d-956f-c89bdd4f8054\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6jpk"
Jul 06 22:23:54.805566 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.801952    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/03309f33-ff47-447d-956f-c89bdd4f8054-ovnkube-config\") pod \"ovnkube-node-t6jpk\" (UID: \"03309f33-ff47-447d-956f-c89bdd4f8054\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6jpk"
Jul 06 22:23:54.805566 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.801973    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e5e3cc00-4650-4700-b30d-17b621fb8d28-etc-selinux\") pod \"aws-ebs-csi-driver-node-b8d5j\" (UID: \"e5e3cc00-4650-4700-b30d-17b621fb8d28\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b8d5j"
Jul 06 22:23:54.805566 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.802001    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9902a376-c450-4867-9c7c-d9e5300bce11-os-release\") pod \"multus-additional-cni-plugins-rzszm\" (UID: \"9902a376-c450-4867-9c7c-d9e5300bce11\") " pod="openshift-multus/multus-additional-cni-plugins-rzszm"
Jul 06 22:23:54.805566 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.802020    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/55b0aff6-b51e-4948-8a6f-04eb6756ab6e-etc-sysctl-d\") pod \"tuned-8dl2h\" (UID: \"55b0aff6-b51e-4948-8a6f-04eb6756ab6e\") " pod="openshift-cluster-node-tuning-operator/tuned-8dl2h"
Jul 06 22:23:54.805566 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.802028    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qbfbg\" (UniqueName: \"kubernetes.io/projected/69b0b78e-e934-433b-bfdb-5fd6d8ba4a29-kube-api-access-qbfbg\") pod \"network-metrics-daemon-bbjgn\" (UID: \"69b0b78e-e934-433b-bfdb-5fd6d8ba4a29\") " pod="openshift-multus/network-metrics-daemon-bbjgn"
Jul 06 22:23:54.805566 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.802043    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9902a376-c450-4867-9c7c-d9e5300bce11-cni-binary-copy\") pod \"multus-additional-cni-plugins-rzszm\" (UID: \"9902a376-c450-4867-9c7c-d9e5300bce11\") " pod="openshift-multus/multus-additional-cni-plugins-rzszm"
Jul 06 22:23:54.805566 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.802054    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/55b0aff6-b51e-4948-8a6f-04eb6756ab6e-etc-modprobe-d\") pod \"tuned-8dl2h\" (UID: \"55b0aff6-b51e-4948-8a6f-04eb6756ab6e\") " pod="openshift-cluster-node-tuning-operator/tuned-8dl2h"
Jul 06 22:23:54.805566 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.802086    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e5e3cc00-4650-4700-b30d-17b621fb8d28-registration-dir\") pod \"aws-ebs-csi-driver-node-b8d5j\" (UID: \"e5e3cc00-4650-4700-b30d-17b621fb8d28\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b8d5j"
Jul 06 22:23:54.805566 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.802483    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e5e3cc00-4650-4700-b30d-17b621fb8d28-device-dir\") pod \"aws-ebs-csi-driver-node-b8d5j\" (UID: \"e5e3cc00-4650-4700-b30d-17b621fb8d28\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b8d5j"
Jul 06 22:23:54.805566 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.802539    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/03309f33-ff47-447d-956f-c89bdd4f8054-host-run-netns\") pod \"ovnkube-node-t6jpk\" (UID: \"03309f33-ff47-447d-956f-c89bdd4f8054\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6jpk"
Jul 06 22:23:54.806092 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.802633    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/03309f33-ff47-447d-956f-c89bdd4f8054-etc-openvswitch\") pod \"ovnkube-node-t6jpk\" (UID: \"03309f33-ff47-447d-956f-c89bdd4f8054\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6jpk"
Jul 06 22:23:54.806092 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.802652    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e5e3cc00-4650-4700-b30d-17b621fb8d28-etc-selinux\") pod \"aws-ebs-csi-driver-node-b8d5j\" (UID: \"e5e3cc00-4650-4700-b30d-17b621fb8d28\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b8d5j"
Jul 06 22:23:54.806092 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.802722    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/03309f33-ff47-447d-956f-c89bdd4f8054-var-lib-openvswitch\") pod \"ovnkube-node-t6jpk\" (UID: \"03309f33-ff47-447d-956f-c89bdd4f8054\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6jpk"
Jul 06 22:23:54.806092 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.802739    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9902a376-c450-4867-9c7c-d9e5300bce11-os-release\") pod \"multus-additional-cni-plugins-rzszm\" (UID: \"9902a376-c450-4867-9c7c-d9e5300bce11\") " pod="openshift-multus/multus-additional-cni-plugins-rzszm"
Jul 06 22:23:54.806092 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.802755    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/03309f33-ff47-447d-956f-c89bdd4f8054-ovnkube-script-lib\") pod \"ovnkube-node-t6jpk\" (UID: \"03309f33-ff47-447d-956f-c89bdd4f8054\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6jpk"
Jul 06 22:23:54.806092 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.802841    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/55b0aff6-b51e-4948-8a6f-04eb6756ab6e-etc-modprobe-d\") pod \"tuned-8dl2h\" (UID: \"55b0aff6-b51e-4948-8a6f-04eb6756ab6e\") " pod="openshift-cluster-node-tuning-operator/tuned-8dl2h"
Jul 06 22:23:54.806092 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.802845    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e5e3cc00-4650-4700-b30d-17b621fb8d28-registration-dir\") pod \"aws-ebs-csi-driver-node-b8d5j\" (UID: \"e5e3cc00-4650-4700-b30d-17b621fb8d28\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b8d5j"
Jul 06 22:23:54.806092 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.803052    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e5e3cc00-4650-4700-b30d-17b621fb8d28-device-dir\") pod \"aws-ebs-csi-driver-node-b8d5j\" (UID: \"e5e3cc00-4650-4700-b30d-17b621fb8d28\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b8d5j"
Jul 06 22:23:54.806092 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.803093    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/55b0aff6-b51e-4948-8a6f-04eb6756ab6e-etc-kubernetes\") pod \"tuned-8dl2h\" (UID: \"55b0aff6-b51e-4948-8a6f-04eb6756ab6e\") " pod="openshift-cluster-node-tuning-operator/tuned-8dl2h"
Jul 06 22:23:54.806092 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.803123    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/55b0aff6-b51e-4948-8a6f-04eb6756ab6e-etc-tuned\") pod \"tuned-8dl2h\" (UID: \"55b0aff6-b51e-4948-8a6f-04eb6756ab6e\") " pod="openshift-cluster-node-tuning-operator/tuned-8dl2h"
Jul 06 22:23:54.806092 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.803294    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/55b0aff6-b51e-4948-8a6f-04eb6756ab6e-etc-kubernetes\") pod \"tuned-8dl2h\" (UID: \"55b0aff6-b51e-4948-8a6f-04eb6756ab6e\") " pod="openshift-cluster-node-tuning-operator/tuned-8dl2h"
Jul 06 22:23:54.806092 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.803417    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/03309f33-ff47-447d-956f-c89bdd4f8054-ovn-node-metrics-cert\") pod \"ovnkube-node-t6jpk\" (UID: \"03309f33-ff47-447d-956f-c89bdd4f8054\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6jpk"
Jul 06 22:23:54.806092 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.803535    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/686a5812-1550-4164-8e13-080cf1953bdb-agent-certs\") pod \"konnectivity-agent-z88v4\" (UID: \"686a5812-1550-4164-8e13-080cf1953bdb\") " pod="kube-system/konnectivity-agent-z88v4"
Jul 06 22:23:54.806092 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.805087    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/55b0aff6-b51e-4948-8a6f-04eb6756ab6e-tmp\") pod \"tuned-8dl2h\" (UID: \"55b0aff6-b51e-4948-8a6f-04eb6756ab6e\") " pod="openshift-cluster-node-tuning-operator/tuned-8dl2h"
Jul 06 22:23:54.806092 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.805613    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/55b0aff6-b51e-4948-8a6f-04eb6756ab6e-etc-tuned\") pod \"tuned-8dl2h\" (UID: \"55b0aff6-b51e-4948-8a6f-04eb6756ab6e\") " pod="openshift-cluster-node-tuning-operator/tuned-8dl2h"
Jul 06 22:23:54.814802 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.814752    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mntz2\" (UniqueName: \"kubernetes.io/projected/e5e3cc00-4650-4700-b30d-17b621fb8d28-kube-api-access-mntz2\") pod \"aws-ebs-csi-driver-node-b8d5j\" (UID: \"e5e3cc00-4650-4700-b30d-17b621fb8d28\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b8d5j"
Jul 06 22:23:54.816568 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.816524    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pcrdh\" (UniqueName: \"kubernetes.io/projected/03309f33-ff47-447d-956f-c89bdd4f8054-kube-api-access-pcrdh\") pod \"ovnkube-node-t6jpk\" (UID: \"03309f33-ff47-447d-956f-c89bdd4f8054\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6jpk"
Jul 06 22:23:54.816690 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.816628    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dgzgf\" (UniqueName: \"kubernetes.io/projected/55b0aff6-b51e-4948-8a6f-04eb6756ab6e-kube-api-access-dgzgf\") pod \"tuned-8dl2h\" (UID: \"55b0aff6-b51e-4948-8a6f-04eb6756ab6e\") " pod="openshift-cluster-node-tuning-operator/tuned-8dl2h"
Jul 06 22:23:54.816756 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.816713    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wtqwf\" (UniqueName: \"kubernetes.io/projected/9902a376-c450-4867-9c7c-d9e5300bce11-kube-api-access-wtqwf\") pod \"multus-additional-cni-plugins-rzszm\" (UID: \"9902a376-c450-4867-9c7c-d9e5300bce11\") " pod="openshift-multus/multus-additional-cni-plugins-rzszm"
Jul 06 22:23:54.816933 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.816904    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qbfbg\" (UniqueName: \"kubernetes.io/projected/69b0b78e-e934-433b-bfdb-5fd6d8ba4a29-kube-api-access-qbfbg\") pod \"network-metrics-daemon-bbjgn\" (UID: \"69b0b78e-e934-433b-bfdb-5fd6d8ba4a29\") " pod="openshift-multus/network-metrics-daemon-bbjgn"
Jul 06 22:23:54.816933 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.816907    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ntg8q\" (UniqueName: \"kubernetes.io/projected/809882c8-1856-4dc4-ac3f-710954ae19c7-kube-api-access-ntg8q\") pod \"iptables-alerter-slmxp\" (UID: \"809882c8-1856-4dc4-ac3f-710954ae19c7\") " pod="openshift-network-operator/iptables-alerter-slmxp"
Jul 06 22:23:54.817101 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.817086    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dd78p\" (UniqueName: \"kubernetes.io/projected/6a3050e9-b393-4767-a9c2-dafdf2a17925-kube-api-access-dd78p\") pod \"node-ca-9l9l5\" (UID: \"6a3050e9-b393-4767-a9c2-dafdf2a17925\") " pod="openshift-image-registry/node-ca-9l9l5"
Jul 06 22:23:54.932733 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.932637    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-tgnb8"
Jul 06 22:23:54.941485 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:54.941456    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c27bdd9_eca8_4ef2_a260_56bb21de6b1e.slice/crio-e8bd5f589a3bc79f5cf575e571c1a371d29c950220aa40bb511308751c3c0bed WatchSource:0}: Error finding container e8bd5f589a3bc79f5cf575e571c1a371d29c950220aa40bb511308751c3c0bed: Status 404 returned error can't find the container with id e8bd5f589a3bc79f5cf575e571c1a371d29c950220aa40bb511308751c3c0bed
Jul 06 22:23:54.947705 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.947687    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-9l9l5"
Jul 06 22:23:54.955034 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:54.955009    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a3050e9_b393_4767_a9c2_dafdf2a17925.slice/crio-6e0a717da2a223c6884e343e7fe7efdd404e3d909f3b2ea9270fe983d67065d3 WatchSource:0}: Error finding container 6e0a717da2a223c6884e343e7fe7efdd404e3d909f3b2ea9270fe983d67065d3: Status 404 returned error can't find the container with id 6e0a717da2a223c6884e343e7fe7efdd404e3d909f3b2ea9270fe983d67065d3
Jul 06 22:23:54.965155 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.965133    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-vs62k"
Jul 06 22:23:54.965155 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.965152    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-t6jpk"
Jul 06 22:23:54.974370 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:54.974344    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79685e2c_e9e0_4ebe_93e6_12dd50c3c75c.slice/crio-27f2154dc7fce76d5882a9610de5b66ae2eaecba8b0214d795ac1a08a05f39ae WatchSource:0}: Error finding container 27f2154dc7fce76d5882a9610de5b66ae2eaecba8b0214d795ac1a08a05f39ae: Status 404 returned error can't find the container with id 27f2154dc7fce76d5882a9610de5b66ae2eaecba8b0214d795ac1a08a05f39ae
Jul 06 22:23:54.977026 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:54.977005    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03309f33_ff47_447d_956f_c89bdd4f8054.slice/crio-9dfe9a6f2738dd26ad2e91d5d7fd37b5b0269a385b49a70668a68e681c276e39 WatchSource:0}: Error finding container 9dfe9a6f2738dd26ad2e91d5d7fd37b5b0269a385b49a70668a68e681c276e39: Status 404 returned error can't find the container with id 9dfe9a6f2738dd26ad2e91d5d7fd37b5b0269a385b49a70668a68e681c276e39
Jul 06 22:23:54.982220 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.982157    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-z88v4"
Jul 06 22:23:54.989714 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:54.989694    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod686a5812_1550_4164_8e13_080cf1953bdb.slice/crio-7f8610fc44185977f57845952c5bdcada2efaee5a1868cb469c74ff1935ed922 WatchSource:0}: Error finding container 7f8610fc44185977f57845952c5bdcada2efaee5a1868cb469c74ff1935ed922: Status 404 returned error can't find the container with id 7f8610fc44185977f57845952c5bdcada2efaee5a1868cb469c74ff1935ed922
Jul 06 22:23:54.994780 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:54.994760    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-rzszm"
Jul 06 22:23:55.001619 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:55.001602    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9902a376_c450_4867_9c7c_d9e5300bce11.slice/crio-87c7c10a668752cf6739555bcb95417b3be80343026465c87b0a8be4b25e2504 WatchSource:0}: Error finding container 87c7c10a668752cf6739555bcb95417b3be80343026465c87b0a8be4b25e2504: Status 404 returned error can't find the container with id 87c7c10a668752cf6739555bcb95417b3be80343026465c87b0a8be4b25e2504
Jul 06 22:23:55.013660 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:55.013640    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-slmxp"
Jul 06 22:23:55.018409 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:55.018392    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b8d5j"
Jul 06 22:23:55.020828 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:55.020749    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod809882c8_1856_4dc4_ac3f_710954ae19c7.slice/crio-00446a9f3191731ebeb8fdd557fed0940f67c17becbd666139fc87958060c2d5 WatchSource:0}: Error finding container 00446a9f3191731ebeb8fdd557fed0940f67c17becbd666139fc87958060c2d5: Status 404 returned error can't find the container with id 00446a9f3191731ebeb8fdd557fed0940f67c17becbd666139fc87958060c2d5
Jul 06 22:23:55.021283 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:55.021268    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-8dl2h"
Jul 06 22:23:55.027100 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:55.027082    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5e3cc00_4650_4700_b30d_17b621fb8d28.slice/crio-fcb8a0a46296f4e56958095be3484a4482a601cf1de038eba7bab6adb691983b WatchSource:0}: Error finding container fcb8a0a46296f4e56958095be3484a4482a601cf1de038eba7bab6adb691983b: Status 404 returned error can't find the container with id fcb8a0a46296f4e56958095be3484a4482a601cf1de038eba7bab6adb691983b
Jul 06 22:23:55.029227 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:23:55.029204    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55b0aff6_b51e_4948_8a6f_04eb6756ab6e.slice/crio-6cc202df8ac13bbaf54e2ca1d5691327a1be2bbd7535a9f0b76da8b9fe62fb9d WatchSource:0}: Error finding container 6cc202df8ac13bbaf54e2ca1d5691327a1be2bbd7535a9f0b76da8b9fe62fb9d: Status 404 returned error can't find the container with id 6cc202df8ac13bbaf54e2ca1d5691327a1be2bbd7535a9f0b76da8b9fe62fb9d
Jul 06 22:23:56.536577 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:56.536545    2563 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 06 22:23:56.593911 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:56.593825    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gvmwj\" (UniqueName: \"kubernetes.io/projected/da82c66a-1f8a-4315-8f45-8b449e2e46b9-kube-api-access-gvmwj\") pod \"network-check-target-6qmpm\" (UID: \"da82c66a-1f8a-4315-8f45-8b449e2e46b9\") " pod="openshift-network-diagnostics/network-check-target-6qmpm"
Jul 06 22:23:56.594067 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:23:56.593983    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 06 22:23:56.594067 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:23:56.594004    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 06 22:23:56.594067 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:23:56.594017    2563 projected.go:196] Error preparing data for projected volume kube-api-access-gvmwj for pod openshift-network-diagnostics/network-check-target-6qmpm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 22:23:56.594250 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:23:56.594077    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/da82c66a-1f8a-4315-8f45-8b449e2e46b9-kube-api-access-gvmwj podName:da82c66a-1f8a-4315-8f45-8b449e2e46b9 nodeName:}" failed. No retries permitted until 2026-07-06 22:23:57.594058979 +0000 UTC m=+3.019105497 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-gvmwj" (UniqueName: "kubernetes.io/projected/da82c66a-1f8a-4315-8f45-8b449e2e46b9-kube-api-access-gvmwj") pod "network-check-target-6qmpm" (UID: "da82c66a-1f8a-4315-8f45-8b449e2e46b9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 22:23:56.694965 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:56.694909    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/aad76575-952c-4b85-b64a-2523529c1cf4-original-pull-secret\") pod \"global-pull-secret-syncer-mc48r\" (UID: \"aad76575-952c-4b85-b64a-2523529c1cf4\") " pod="kube-system/global-pull-secret-syncer-mc48r"
Jul 06 22:23:56.695116 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:56.695025    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/69b0b78e-e934-433b-bfdb-5fd6d8ba4a29-metrics-certs\") pod \"network-metrics-daemon-bbjgn\" (UID: \"69b0b78e-e934-433b-bfdb-5fd6d8ba4a29\") " pod="openshift-multus/network-metrics-daemon-bbjgn"
Jul 06 22:23:56.695207 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:23:56.695190    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 22:23:56.695268 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:23:56.695255    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/69b0b78e-e934-433b-bfdb-5fd6d8ba4a29-metrics-certs podName:69b0b78e-e934-433b-bfdb-5fd6d8ba4a29 nodeName:}" failed. No retries permitted until 2026-07-06 22:23:57.695235563 +0000 UTC m=+3.120282087 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/69b0b78e-e934-433b-bfdb-5fd6d8ba4a29-metrics-certs") pod "network-metrics-daemon-bbjgn" (UID: "69b0b78e-e934-433b-bfdb-5fd6d8ba4a29") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 22:23:56.695349 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:23:56.695333    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 06 22:23:56.695405 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:23:56.695376    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aad76575-952c-4b85-b64a-2523529c1cf4-original-pull-secret podName:aad76575-952c-4b85-b64a-2523529c1cf4 nodeName:}" failed. No retries permitted until 2026-07-06 22:23:57.695364773 +0000 UTC m=+3.120411306 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/aad76575-952c-4b85-b64a-2523529c1cf4-original-pull-secret") pod "global-pull-secret-syncer-mc48r" (UID: "aad76575-952c-4b85-b64a-2523529c1cf4") : object "kube-system"/"original-pull-secret" not registered
Jul 06 22:23:56.831381 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:56.831330    2563 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 06 22:23:57.019857 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:57.019746    2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-05 22:18:56 +0000 UTC" deadline="2028-02-11 01:19:46.46879799 +0000 UTC"
Jul 06 22:23:57.019857 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:57.019816    2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14018h55m49.44898723s"
Jul 06 22:23:57.091591 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:57.091475    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b8d5j" event={"ID":"e5e3cc00-4650-4700-b30d-17b621fb8d28","Type":"ContainerStarted","Data":"fcb8a0a46296f4e56958095be3484a4482a601cf1de038eba7bab6adb691983b"}
Jul 06 22:23:57.107522 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:57.107488    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-vs62k" event={"ID":"79685e2c-e9e0-4ebe-93e6-12dd50c3c75c","Type":"ContainerStarted","Data":"27f2154dc7fce76d5882a9610de5b66ae2eaecba8b0214d795ac1a08a05f39ae"}
Jul 06 22:23:57.118367 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:57.118331    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-9l9l5" event={"ID":"6a3050e9-b393-4767-a9c2-dafdf2a17925","Type":"ContainerStarted","Data":"6e0a717da2a223c6884e343e7fe7efdd404e3d909f3b2ea9270fe983d67065d3"}
Jul 06 22:23:57.124962 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:57.124900    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-tgnb8" event={"ID":"9c27bdd9-eca8-4ef2-a260-56bb21de6b1e","Type":"ContainerStarted","Data":"e8bd5f589a3bc79f5cf575e571c1a371d29c950220aa40bb511308751c3c0bed"}
Jul 06 22:23:57.130083 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:57.130056    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-105.ec2.internal" event={"ID":"a7e777c6384fb2d7e87381cc5d382585","Type":"ContainerStarted","Data":"2471430135b24cea418b887512cb4a77fc3559d77c3f8fc2ce56baf7592b933b"}
Jul 06 22:23:57.133774 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:57.133744    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-105.ec2.internal" event={"ID":"80e55e8f9c22a5bcba25f5b76deba1dd","Type":"ContainerStarted","Data":"aab95d1c2871b7f6f6749f2b503fa8e0f4742212d8b944ad45365682aa610a87"}
Jul 06 22:23:57.140225 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:57.140200    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-8dl2h" event={"ID":"55b0aff6-b51e-4948-8a6f-04eb6756ab6e","Type":"ContainerStarted","Data":"6cc202df8ac13bbaf54e2ca1d5691327a1be2bbd7535a9f0b76da8b9fe62fb9d"}
Jul 06 22:23:57.152847 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:57.152821    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-slmxp" event={"ID":"809882c8-1856-4dc4-ac3f-710954ae19c7","Type":"ContainerStarted","Data":"00446a9f3191731ebeb8fdd557fed0940f67c17becbd666139fc87958060c2d5"}
Jul 06 22:23:57.154216 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:57.154196    2563 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 06 22:23:57.159703 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:57.159676    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rzszm" event={"ID":"9902a376-c450-4867-9c7c-d9e5300bce11","Type":"ContainerStarted","Data":"87c7c10a668752cf6739555bcb95417b3be80343026465c87b0a8be4b25e2504"}
Jul 06 22:23:57.164120 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:57.164091    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-z88v4" event={"ID":"686a5812-1550-4164-8e13-080cf1953bdb","Type":"ContainerStarted","Data":"7f8610fc44185977f57845952c5bdcada2efaee5a1868cb469c74ff1935ed922"}
Jul 06 22:23:57.169716 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:57.169661    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t6jpk" event={"ID":"03309f33-ff47-447d-956f-c89bdd4f8054","Type":"ContainerStarted","Data":"9dfe9a6f2738dd26ad2e91d5d7fd37b5b0269a385b49a70668a68e681c276e39"}
Jul 06 22:23:57.602364 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:57.601695    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gvmwj\" (UniqueName: \"kubernetes.io/projected/da82c66a-1f8a-4315-8f45-8b449e2e46b9-kube-api-access-gvmwj\") pod \"network-check-target-6qmpm\" (UID: \"da82c66a-1f8a-4315-8f45-8b449e2e46b9\") " pod="openshift-network-diagnostics/network-check-target-6qmpm"
Jul 06 22:23:57.602364 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:23:57.601877    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 06 22:23:57.602364 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:23:57.601895    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 06 22:23:57.602364 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:23:57.601910    2563 projected.go:196] Error preparing data for projected volume kube-api-access-gvmwj for pod openshift-network-diagnostics/network-check-target-6qmpm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 22:23:57.602364 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:23:57.601968    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/da82c66a-1f8a-4315-8f45-8b449e2e46b9-kube-api-access-gvmwj podName:da82c66a-1f8a-4315-8f45-8b449e2e46b9 nodeName:}" failed. No retries permitted until 2026-07-06 22:23:59.601950917 +0000 UTC m=+5.026997442 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-gvmwj" (UniqueName: "kubernetes.io/projected/da82c66a-1f8a-4315-8f45-8b449e2e46b9-kube-api-access-gvmwj") pod "network-check-target-6qmpm" (UID: "da82c66a-1f8a-4315-8f45-8b449e2e46b9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 22:23:57.702120 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:57.702079    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/aad76575-952c-4b85-b64a-2523529c1cf4-original-pull-secret\") pod \"global-pull-secret-syncer-mc48r\" (UID: \"aad76575-952c-4b85-b64a-2523529c1cf4\") " pod="kube-system/global-pull-secret-syncer-mc48r"
Jul 06 22:23:57.702309 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:57.702162    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/69b0b78e-e934-433b-bfdb-5fd6d8ba4a29-metrics-certs\") pod \"network-metrics-daemon-bbjgn\" (UID: \"69b0b78e-e934-433b-bfdb-5fd6d8ba4a29\") " pod="openshift-multus/network-metrics-daemon-bbjgn"
Jul 06 22:23:57.702376 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:23:57.702344    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 22:23:57.702425 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:23:57.702412    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/69b0b78e-e934-433b-bfdb-5fd6d8ba4a29-metrics-certs podName:69b0b78e-e934-433b-bfdb-5fd6d8ba4a29 nodeName:}" failed. No retries permitted until 2026-07-06 22:23:59.702390957 +0000 UTC m=+5.127437494 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/69b0b78e-e934-433b-bfdb-5fd6d8ba4a29-metrics-certs") pod "network-metrics-daemon-bbjgn" (UID: "69b0b78e-e934-433b-bfdb-5fd6d8ba4a29") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 22:23:57.702828 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:23:57.702805    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 06 22:23:57.702934 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:23:57.702854    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aad76575-952c-4b85-b64a-2523529c1cf4-original-pull-secret podName:aad76575-952c-4b85-b64a-2523529c1cf4 nodeName:}" failed. No retries permitted until 2026-07-06 22:23:59.702839735 +0000 UTC m=+5.127886269 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/aad76575-952c-4b85-b64a-2523529c1cf4-original-pull-secret") pod "global-pull-secret-syncer-mc48r" (UID: "aad76575-952c-4b85-b64a-2523529c1cf4") : object "kube-system"/"original-pull-secret" not registered
Jul 06 22:23:58.020732 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:58.020673    2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-05 22:18:56 +0000 UTC" deadline="2028-02-02 20:31:52.781769588 +0000 UTC"
Jul 06 22:23:58.020732 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:58.020723    2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13822h7m54.761050904s"
Jul 06 22:23:58.074384 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:58.074353    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bbjgn"
Jul 06 22:23:58.074545 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:23:58.074481    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bbjgn" podUID="69b0b78e-e934-433b-bfdb-5fd6d8ba4a29"
Jul 06 22:23:58.074916 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:58.074878    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6qmpm"
Jul 06 22:23:58.075032 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:23:58.074997    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6qmpm" podUID="da82c66a-1f8a-4315-8f45-8b449e2e46b9"
Jul 06 22:23:58.075102 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:58.075061    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mc48r"
Jul 06 22:23:58.075152 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:23:58.075135    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mc48r" podUID="aad76575-952c-4b85-b64a-2523529c1cf4"
Jul 06 22:23:58.642291 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:58.642054    2563 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 06 22:23:59.620270 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:59.620234    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gvmwj\" (UniqueName: \"kubernetes.io/projected/da82c66a-1f8a-4315-8f45-8b449e2e46b9-kube-api-access-gvmwj\") pod \"network-check-target-6qmpm\" (UID: \"da82c66a-1f8a-4315-8f45-8b449e2e46b9\") " pod="openshift-network-diagnostics/network-check-target-6qmpm"
Jul 06 22:23:59.620712 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:23:59.620426    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 06 22:23:59.620712 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:23:59.620445    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 06 22:23:59.620712 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:23:59.620457    2563 projected.go:196] Error preparing data for projected volume kube-api-access-gvmwj for pod openshift-network-diagnostics/network-check-target-6qmpm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 22:23:59.620712 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:23:59.620515    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/da82c66a-1f8a-4315-8f45-8b449e2e46b9-kube-api-access-gvmwj podName:da82c66a-1f8a-4315-8f45-8b449e2e46b9 nodeName:}" failed. No retries permitted until 2026-07-06 22:24:03.620495636 +0000 UTC m=+9.045542175 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-gvmwj" (UniqueName: "kubernetes.io/projected/da82c66a-1f8a-4315-8f45-8b449e2e46b9-kube-api-access-gvmwj") pod "network-check-target-6qmpm" (UID: "da82c66a-1f8a-4315-8f45-8b449e2e46b9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 22:23:59.721360 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:59.721325    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/aad76575-952c-4b85-b64a-2523529c1cf4-original-pull-secret\") pod \"global-pull-secret-syncer-mc48r\" (UID: \"aad76575-952c-4b85-b64a-2523529c1cf4\") " pod="kube-system/global-pull-secret-syncer-mc48r"
Jul 06 22:23:59.721529 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:23:59.721394    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/69b0b78e-e934-433b-bfdb-5fd6d8ba4a29-metrics-certs\") pod \"network-metrics-daemon-bbjgn\" (UID: \"69b0b78e-e934-433b-bfdb-5fd6d8ba4a29\") " pod="openshift-multus/network-metrics-daemon-bbjgn"
Jul 06 22:23:59.721529 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:23:59.721526    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 22:23:59.721639 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:23:59.721583    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/69b0b78e-e934-433b-bfdb-5fd6d8ba4a29-metrics-certs podName:69b0b78e-e934-433b-bfdb-5fd6d8ba4a29 nodeName:}" failed. No retries permitted until 2026-07-06 22:24:03.721565569 +0000 UTC m=+9.146612095 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/69b0b78e-e934-433b-bfdb-5fd6d8ba4a29-metrics-certs") pod "network-metrics-daemon-bbjgn" (UID: "69b0b78e-e934-433b-bfdb-5fd6d8ba4a29") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 22:23:59.721976 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:23:59.721955    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 06 22:23:59.722051 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:23:59.722014    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aad76575-952c-4b85-b64a-2523529c1cf4-original-pull-secret podName:aad76575-952c-4b85-b64a-2523529c1cf4 nodeName:}" failed. No retries permitted until 2026-07-06 22:24:03.721999757 +0000 UTC m=+9.147046279 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/aad76575-952c-4b85-b64a-2523529c1cf4-original-pull-secret") pod "global-pull-secret-syncer-mc48r" (UID: "aad76575-952c-4b85-b64a-2523529c1cf4") : object "kube-system"/"original-pull-secret" not registered
Jul 06 22:24:00.074499 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:00.074425    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bbjgn"
Jul 06 22:24:00.074637 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:24:00.074572    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bbjgn" podUID="69b0b78e-e934-433b-bfdb-5fd6d8ba4a29"
Jul 06 22:24:00.074920 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:00.074899    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6qmpm"
Jul 06 22:24:00.075021 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:24:00.075000    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6qmpm" podUID="da82c66a-1f8a-4315-8f45-8b449e2e46b9"
Jul 06 22:24:00.075065 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:00.075051    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mc48r"
Jul 06 22:24:00.075139 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:24:00.075121    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mc48r" podUID="aad76575-952c-4b85-b64a-2523529c1cf4"
Jul 06 22:24:02.074618 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:02.074581    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mc48r"
Jul 06 22:24:02.074618 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:02.074622    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6qmpm"
Jul 06 22:24:02.075148 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:02.074587    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bbjgn"
Jul 06 22:24:02.075148 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:24:02.074720    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mc48r" podUID="aad76575-952c-4b85-b64a-2523529c1cf4"
Jul 06 22:24:02.075148 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:24:02.074795    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bbjgn" podUID="69b0b78e-e934-433b-bfdb-5fd6d8ba4a29"
Jul 06 22:24:02.075148 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:24:02.074859    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6qmpm" podUID="da82c66a-1f8a-4315-8f45-8b449e2e46b9"
Jul 06 22:24:03.654806 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:03.654718    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gvmwj\" (UniqueName: \"kubernetes.io/projected/da82c66a-1f8a-4315-8f45-8b449e2e46b9-kube-api-access-gvmwj\") pod \"network-check-target-6qmpm\" (UID: \"da82c66a-1f8a-4315-8f45-8b449e2e46b9\") " pod="openshift-network-diagnostics/network-check-target-6qmpm"
Jul 06 22:24:03.655323 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:24:03.654893    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 06 22:24:03.655323 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:24:03.654924    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 06 22:24:03.655323 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:24:03.654941    2563 projected.go:196] Error preparing data for projected volume kube-api-access-gvmwj for pod openshift-network-diagnostics/network-check-target-6qmpm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 22:24:03.655323 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:24:03.655009    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/da82c66a-1f8a-4315-8f45-8b449e2e46b9-kube-api-access-gvmwj podName:da82c66a-1f8a-4315-8f45-8b449e2e46b9 nodeName:}" failed. No retries permitted until 2026-07-06 22:24:11.654989693 +0000 UTC m=+17.080036231 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-gvmwj" (UniqueName: "kubernetes.io/projected/da82c66a-1f8a-4315-8f45-8b449e2e46b9-kube-api-access-gvmwj") pod "network-check-target-6qmpm" (UID: "da82c66a-1f8a-4315-8f45-8b449e2e46b9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 22:24:03.755238 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:03.755203    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/69b0b78e-e934-433b-bfdb-5fd6d8ba4a29-metrics-certs\") pod \"network-metrics-daemon-bbjgn\" (UID: \"69b0b78e-e934-433b-bfdb-5fd6d8ba4a29\") " pod="openshift-multus/network-metrics-daemon-bbjgn"
Jul 06 22:24:03.755437 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:03.755277    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/aad76575-952c-4b85-b64a-2523529c1cf4-original-pull-secret\") pod \"global-pull-secret-syncer-mc48r\" (UID: \"aad76575-952c-4b85-b64a-2523529c1cf4\") " pod="kube-system/global-pull-secret-syncer-mc48r"
Jul 06 22:24:03.755437 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:24:03.755386    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 06 22:24:03.755551 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:24:03.755442    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aad76575-952c-4b85-b64a-2523529c1cf4-original-pull-secret podName:aad76575-952c-4b85-b64a-2523529c1cf4 nodeName:}" failed. No retries permitted until 2026-07-06 22:24:11.755423613 +0000 UTC m=+17.180470134 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/aad76575-952c-4b85-b64a-2523529c1cf4-original-pull-secret") pod "global-pull-secret-syncer-mc48r" (UID: "aad76575-952c-4b85-b64a-2523529c1cf4") : object "kube-system"/"original-pull-secret" not registered
Jul 06 22:24:03.755871 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:24:03.755785    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 22:24:03.755871 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:24:03.755828    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/69b0b78e-e934-433b-bfdb-5fd6d8ba4a29-metrics-certs podName:69b0b78e-e934-433b-bfdb-5fd6d8ba4a29 nodeName:}" failed. No retries permitted until 2026-07-06 22:24:11.755816522 +0000 UTC m=+17.180863041 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/69b0b78e-e934-433b-bfdb-5fd6d8ba4a29-metrics-certs") pod "network-metrics-daemon-bbjgn" (UID: "69b0b78e-e934-433b-bfdb-5fd6d8ba4a29") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 22:24:04.074957 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:04.074289    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mc48r"
Jul 06 22:24:04.074957 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:24:04.074420    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mc48r" podUID="aad76575-952c-4b85-b64a-2523529c1cf4"
Jul 06 22:24:04.074957 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:04.074293    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6qmpm"
Jul 06 22:24:04.074957 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:24:04.074533    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6qmpm" podUID="da82c66a-1f8a-4315-8f45-8b449e2e46b9"
Jul 06 22:24:04.074957 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:04.074289    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bbjgn"
Jul 06 22:24:04.074957 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:24:04.074608    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bbjgn" podUID="69b0b78e-e934-433b-bfdb-5fd6d8ba4a29"
Jul 06 22:24:06.074696 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:06.074664    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mc48r"
Jul 06 22:24:06.075127 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:06.074672    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bbjgn"
Jul 06 22:24:06.075127 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:24:06.074776    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mc48r" podUID="aad76575-952c-4b85-b64a-2523529c1cf4"
Jul 06 22:24:06.075127 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:24:06.074911    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bbjgn" podUID="69b0b78e-e934-433b-bfdb-5fd6d8ba4a29"
Jul 06 22:24:06.075127 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:06.074678    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6qmpm"
Jul 06 22:24:06.075127 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:24:06.075010    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6qmpm" podUID="da82c66a-1f8a-4315-8f45-8b449e2e46b9"
Jul 06 22:24:08.074640 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:08.074556    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bbjgn"
Jul 06 22:24:08.074640 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:08.074590    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6qmpm"
Jul 06 22:24:08.075072 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:08.074557    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mc48r"
Jul 06 22:24:08.075072 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:24:08.074700    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bbjgn" podUID="69b0b78e-e934-433b-bfdb-5fd6d8ba4a29"
Jul 06 22:24:08.075072 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:24:08.074776    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mc48r" podUID="aad76575-952c-4b85-b64a-2523529c1cf4"
Jul 06 22:24:08.075072 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:24:08.074854    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6qmpm" podUID="da82c66a-1f8a-4315-8f45-8b449e2e46b9"
Jul 06 22:24:10.074625 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:10.074367    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bbjgn"
Jul 06 22:24:10.075092 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:10.074396    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6qmpm"
Jul 06 22:24:10.075092 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:24:10.074745    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bbjgn" podUID="69b0b78e-e934-433b-bfdb-5fd6d8ba4a29"
Jul 06 22:24:10.075092 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:10.074400    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mc48r"
Jul 06 22:24:10.075092 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:24:10.074790    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6qmpm" podUID="da82c66a-1f8a-4315-8f45-8b449e2e46b9"
Jul 06 22:24:10.075092 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:24:10.074886    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mc48r" podUID="aad76575-952c-4b85-b64a-2523529c1cf4"
Jul 06 22:24:11.721944 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:11.721914    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gvmwj\" (UniqueName: \"kubernetes.io/projected/da82c66a-1f8a-4315-8f45-8b449e2e46b9-kube-api-access-gvmwj\") pod \"network-check-target-6qmpm\" (UID: \"da82c66a-1f8a-4315-8f45-8b449e2e46b9\") " pod="openshift-network-diagnostics/network-check-target-6qmpm"
Jul 06 22:24:11.722340 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:24:11.722067    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 06 22:24:11.722340 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:24:11.722085    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 06 22:24:11.722340 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:24:11.722095    2563 projected.go:196] Error preparing data for projected volume kube-api-access-gvmwj for pod openshift-network-diagnostics/network-check-target-6qmpm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 22:24:11.722340 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:24:11.722142    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/da82c66a-1f8a-4315-8f45-8b449e2e46b9-kube-api-access-gvmwj podName:da82c66a-1f8a-4315-8f45-8b449e2e46b9 nodeName:}" failed. No retries permitted until 2026-07-06 22:24:27.72212923 +0000 UTC m=+33.147175754 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-gvmwj" (UniqueName: "kubernetes.io/projected/da82c66a-1f8a-4315-8f45-8b449e2e46b9-kube-api-access-gvmwj") pod "network-check-target-6qmpm" (UID: "da82c66a-1f8a-4315-8f45-8b449e2e46b9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 22:24:11.822429 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:11.822391    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/aad76575-952c-4b85-b64a-2523529c1cf4-original-pull-secret\") pod \"global-pull-secret-syncer-mc48r\" (UID: \"aad76575-952c-4b85-b64a-2523529c1cf4\") " pod="kube-system/global-pull-secret-syncer-mc48r"
Jul 06 22:24:11.822429 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:11.822436    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/69b0b78e-e934-433b-bfdb-5fd6d8ba4a29-metrics-certs\") pod \"network-metrics-daemon-bbjgn\" (UID: \"69b0b78e-e934-433b-bfdb-5fd6d8ba4a29\") " pod="openshift-multus/network-metrics-daemon-bbjgn"
Jul 06 22:24:11.822695 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:24:11.822545    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 22:24:11.822695 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:24:11.822549    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 06 22:24:11.822695 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:24:11.822598    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/69b0b78e-e934-433b-bfdb-5fd6d8ba4a29-metrics-certs podName:69b0b78e-e934-433b-bfdb-5fd6d8ba4a29 nodeName:}" failed. No retries permitted until 2026-07-06 22:24:27.822582222 +0000 UTC m=+33.247628742 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/69b0b78e-e934-433b-bfdb-5fd6d8ba4a29-metrics-certs") pod "network-metrics-daemon-bbjgn" (UID: "69b0b78e-e934-433b-bfdb-5fd6d8ba4a29") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 22:24:11.822695 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:24:11.822618    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aad76575-952c-4b85-b64a-2523529c1cf4-original-pull-secret podName:aad76575-952c-4b85-b64a-2523529c1cf4 nodeName:}" failed. No retries permitted until 2026-07-06 22:24:27.822604524 +0000 UTC m=+33.247651047 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/aad76575-952c-4b85-b64a-2523529c1cf4-original-pull-secret") pod "global-pull-secret-syncer-mc48r" (UID: "aad76575-952c-4b85-b64a-2523529c1cf4") : object "kube-system"/"original-pull-secret" not registered
Jul 06 22:24:12.073992 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:12.073898    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mc48r"
Jul 06 22:24:12.073992 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:12.073945    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bbjgn"
Jul 06 22:24:12.074234 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:24:12.074020    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mc48r" podUID="aad76575-952c-4b85-b64a-2523529c1cf4"
Jul 06 22:24:12.074234 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:12.074087    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6qmpm"
Jul 06 22:24:12.074234 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:24:12.074202    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6qmpm" podUID="da82c66a-1f8a-4315-8f45-8b449e2e46b9"
Jul 06 22:24:12.074339 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:24:12.074291    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bbjgn" podUID="69b0b78e-e934-433b-bfdb-5fd6d8ba4a29"
Jul 06 22:24:14.074931 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:14.074894    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mc48r"
Jul 06 22:24:14.075439 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:14.074899    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bbjgn"
Jul 06 22:24:14.075439 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:24:14.075026    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mc48r" podUID="aad76575-952c-4b85-b64a-2523529c1cf4"
Jul 06 22:24:14.075439 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:14.074900    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6qmpm"
Jul 06 22:24:14.075439 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:24:14.075089    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bbjgn" podUID="69b0b78e-e934-433b-bfdb-5fd6d8ba4a29"
Jul 06 22:24:14.075439 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:24:14.075216    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6qmpm" podUID="da82c66a-1f8a-4315-8f45-8b449e2e46b9"
Jul 06 22:24:16.074774 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:16.074736    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bbjgn"
Jul 06 22:24:16.075186 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:16.074736    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6qmpm"
Jul 06 22:24:16.075186 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:24:16.074856    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bbjgn" podUID="69b0b78e-e934-433b-bfdb-5fd6d8ba4a29"
Jul 06 22:24:16.075186 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:16.074736    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mc48r"
Jul 06 22:24:16.075186 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:24:16.074926    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6qmpm" podUID="da82c66a-1f8a-4315-8f45-8b449e2e46b9"
Jul 06 22:24:16.075186 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:24:16.074990    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mc48r" podUID="aad76575-952c-4b85-b64a-2523529c1cf4"
Jul 06 22:24:17.213867 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:17.213629    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-vs62k" event={"ID":"79685e2c-e9e0-4ebe-93e6-12dd50c3c75c","Type":"ContainerStarted","Data":"96a44c1045088f2f80f8d274e717a8f581e86851499662ba2cf9118785861b54"}
Jul 06 22:24:17.215216 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:17.215194    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-105.ec2.internal" event={"ID":"80e55e8f9c22a5bcba25f5b76deba1dd","Type":"ContainerStarted","Data":"2da7b21a09b584d2606f2f9f405fbf4cad27243043be4c3434f3f2229d917da8"}
Jul 06 22:24:17.217942 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:17.217914    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-8dl2h" event={"ID":"55b0aff6-b51e-4948-8a6f-04eb6756ab6e","Type":"ContainerStarted","Data":"6ba1733adcb8760919f4a62cf8787020113901387c017fa0c40a7841e6f0d893"}
Jul 06 22:24:17.220746 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:17.220725    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-t6jpk_03309f33-ff47-447d-956f-c89bdd4f8054/ovn-acl-logging/0.log"
Jul 06 22:24:17.221017 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:17.220996    2563 generic.go:356] "Generic (PLEG): container finished" podID="03309f33-ff47-447d-956f-c89bdd4f8054" containerID="ccd5164605b35e284de8e0137cf0cb2053522c0a17944a7da0d972219b5290ec" exitCode=1
Jul 06 22:24:17.221088 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:17.221027    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t6jpk" event={"ID":"03309f33-ff47-447d-956f-c89bdd4f8054","Type":"ContainerStarted","Data":"faf5b36e45b8fabe55698c79f91647acfa4a40e21c0e4ccad745eb0c607c6229"}
Jul 06 22:24:17.221088 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:17.221079    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t6jpk" event={"ID":"03309f33-ff47-447d-956f-c89bdd4f8054","Type":"ContainerStarted","Data":"c7e15f2ac1d4a35570a2cf663937a5f2aace6d18598da491ec4184812071eff5"}
Jul 06 22:24:17.221227 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:17.221109    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t6jpk" event={"ID":"03309f33-ff47-447d-956f-c89bdd4f8054","Type":"ContainerDied","Data":"ccd5164605b35e284de8e0137cf0cb2053522c0a17944a7da0d972219b5290ec"}
Jul 06 22:24:17.221227 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:17.221147    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t6jpk" event={"ID":"03309f33-ff47-447d-956f-c89bdd4f8054","Type":"ContainerStarted","Data":"ee75ea616c8d5c37051e1260cca099078539edd7bda64e5f41264a62f8c6c444"}
Jul 06 22:24:17.246127 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:17.246077    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-vs62k" podStartSLOduration=2.183670732 podStartE2EDuration="22.246061162s" totalImagesPullingTime="20.06239043s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 22:23:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:24:17.245833334 +0000 UTC m=+22.670879887" watchObservedRunningTime="2026-07-06 22:24:17.246061162 +0000 UTC m=+22.671107702"
Jul 06 22:24:17.266956 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:17.266904    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-8dl2h" podStartSLOduration=2.270829345 podStartE2EDuration="22.266888515s" totalImagesPullingTime="19.99605917s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 22:23:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:24:17.266722951 +0000 UTC m=+22.691769493" watchObservedRunningTime="2026-07-06 22:24:17.266888515 +0000 UTC m=+22.691935038"
Jul 06 22:24:17.280277 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:17.280235    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-105.ec2.internal" podStartSLOduration=22.280219249 podStartE2EDuration="22.280219249s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 22:23:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:24:17.279887068 +0000 UTC m=+22.704933609" watchObservedRunningTime="2026-07-06 22:24:17.280219249 +0000 UTC m=+22.705265789"
Jul 06 22:24:18.074321 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:18.074115    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mc48r"
Jul 06 22:24:18.074426 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:18.074192    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bbjgn"
Jul 06 22:24:18.074426 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:24:18.074395    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mc48r" podUID="aad76575-952c-4b85-b64a-2523529c1cf4"
Jul 06 22:24:18.074541 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:24:18.074511    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bbjgn" podUID="69b0b78e-e934-433b-bfdb-5fd6d8ba4a29"
Jul 06 22:24:18.074541 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:18.074192    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6qmpm"
Jul 06 22:24:18.074646 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:24:18.074629    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6qmpm" podUID="da82c66a-1f8a-4315-8f45-8b449e2e46b9"
Jul 06 22:24:18.227279 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:18.227245    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-9l9l5" event={"ID":"6a3050e9-b393-4767-a9c2-dafdf2a17925","Type":"ContainerStarted","Data":"7add3640a83a89a18f9c8a04a4df81d2085f0c4f7d99680da7e7b123729a1d02"}
Jul 06 22:24:18.228726 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:18.228700    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-tgnb8" event={"ID":"9c27bdd9-eca8-4ef2-a260-56bb21de6b1e","Type":"ContainerStarted","Data":"34aeefd1b1907d449f14c4c7ed4d0ccb01a9bf1374b5c1d5d2ffdfe083f9af0a"}
Jul 06 22:24:18.230131 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:18.230107    2563 generic.go:356] "Generic (PLEG): container finished" podID="a7e777c6384fb2d7e87381cc5d382585" containerID="5021ef4ac340c7677d4ebd0746351b321a45c1add3bd156b18c7922317770438" exitCode=0
Jul 06 22:24:18.230256 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:18.230201    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-105.ec2.internal" event={"ID":"a7e777c6384fb2d7e87381cc5d382585","Type":"ContainerDied","Data":"5021ef4ac340c7677d4ebd0746351b321a45c1add3bd156b18c7922317770438"}
Jul 06 22:24:18.231622 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:18.231601    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-slmxp" event={"ID":"809882c8-1856-4dc4-ac3f-710954ae19c7","Type":"ContainerStarted","Data":"a32f1a451423c58536fc55be0b8f40edddd9815ba56d83a5e24a778929600516"}
Jul 06 22:24:18.233023 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:18.233004    2563 generic.go:356] "Generic (PLEG): container finished" podID="9902a376-c450-4867-9c7c-d9e5300bce11" containerID="42afbe7c2bd9dacd0078267a541bde3796425eb9d330dae7b9e2efee447705af" exitCode=0
Jul 06 22:24:18.233110 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:18.233065    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rzszm" event={"ID":"9902a376-c450-4867-9c7c-d9e5300bce11","Type":"ContainerDied","Data":"42afbe7c2bd9dacd0078267a541bde3796425eb9d330dae7b9e2efee447705af"}
Jul 06 22:24:18.234584 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:18.234560    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-z88v4" event={"ID":"686a5812-1550-4164-8e13-080cf1953bdb","Type":"ContainerStarted","Data":"6d5ad76611348ca2e2e6d58895c4e95c487842f8bd1fee1fb4de8e873fb72596"}
Jul 06 22:24:18.239550 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:18.239535    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-t6jpk_03309f33-ff47-447d-956f-c89bdd4f8054/ovn-acl-logging/0.log"
Jul 06 22:24:18.239884 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:18.239868    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t6jpk" event={"ID":"03309f33-ff47-447d-956f-c89bdd4f8054","Type":"ContainerStarted","Data":"c971dd98e9e6a1aaec17b09383a366abb048f591a8d9141a16db0aea1ba2bc05"}
Jul 06 22:24:18.239951 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:18.239917    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t6jpk" event={"ID":"03309f33-ff47-447d-956f-c89bdd4f8054","Type":"ContainerStarted","Data":"ad3eb32c3b99a102aa75da641e2046b096a43c9f7d40bee48b60d63e350a2d11"}
Jul 06 22:24:18.241235 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:18.241216    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b8d5j" event={"ID":"e5e3cc00-4650-4700-b30d-17b621fb8d28","Type":"ContainerStarted","Data":"94e63191e83e8676dc42e6707a52094c5d5dd3ef182ea20b86f66fc23af42b1a"}
Jul 06 22:24:18.242585 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:18.242549    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-9l9l5" podStartSLOduration=3.174970162 podStartE2EDuration="23.24253837s" totalImagesPullingTime="20.067568208s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 22:23:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:24:18.242296743 +0000 UTC m=+23.667343287" watchObservedRunningTime="2026-07-06 22:24:18.24253837 +0000 UTC m=+23.667584926"
Jul 06 22:24:18.260340 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:18.260268    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-z88v4" podStartSLOduration=3.227404149 podStartE2EDuration="23.260256493s" totalImagesPullingTime="20.032852344s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:23:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:24:18.260051153 +0000 UTC m=+23.685097694" watchObservedRunningTime="2026-07-06 22:24:18.260256493 +0000 UTC m=+23.685303033"
Jul 06 22:24:18.289689 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:18.289651    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-tgnb8" podStartSLOduration=7.964945994 podStartE2EDuration="23.289639637s" totalImagesPullingTime="15.324693643s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 22:23:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:24:18.288415315 +0000 UTC m=+23.713461856" watchObservedRunningTime="2026-07-06 22:24:18.289639637 +0000 UTC m=+23.714686178"
Jul 06 22:24:18.352584 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:18.352538    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-slmxp" podStartSLOduration=3.34915911 podStartE2EDuration="23.352523783s" totalImagesPullingTime="20.003364673s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 22:23:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:24:18.352347102 +0000 UTC m=+23.777393653" watchObservedRunningTime="2026-07-06 22:24:18.352523783 +0000 UTC m=+23.777570324"
Jul 06 22:24:19.268428 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:19.268405    2563 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 06 22:24:19.892287 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:19.892221    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-z88v4"
Jul 06 22:24:19.892998 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:19.892976    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-z88v4"
Jul 06 22:24:20.058058 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:20.057955    2563 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-06T22:24:19.268422823Z","UUID":"bd1711c5-6c2e-474a-b526-b2ec24759bc9","Handler":null,"Name":"","Endpoint":""}
Jul 06 22:24:20.059473 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:20.059448    2563 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 06 22:24:20.059605 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:20.059491    2563 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 06 22:24:20.074212 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:20.074188    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mc48r"
Jul 06 22:24:20.074338 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:24:20.074296    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mc48r" podUID="aad76575-952c-4b85-b64a-2523529c1cf4"
Jul 06 22:24:20.074641 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:20.074624    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bbjgn"
Jul 06 22:24:20.074767 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:24:20.074746    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bbjgn" podUID="69b0b78e-e934-433b-bfdb-5fd6d8ba4a29"
Jul 06 22:24:20.074837 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:20.074778    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6qmpm"
Jul 06 22:24:20.074892 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:24:20.074834    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6qmpm" podUID="da82c66a-1f8a-4315-8f45-8b449e2e46b9"
Jul 06 22:24:20.249095 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:20.249069    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-t6jpk_03309f33-ff47-447d-956f-c89bdd4f8054/ovn-acl-logging/0.log"
Jul 06 22:24:20.249522 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:20.249493    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t6jpk" event={"ID":"03309f33-ff47-447d-956f-c89bdd4f8054","Type":"ContainerStarted","Data":"39cc9cf0b4f10b07dce91fafd1fe07565ad642e8d032ade19d9093a827a97109"}
Jul 06 22:24:20.251316 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:20.251289    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b8d5j" event={"ID":"e5e3cc00-4650-4700-b30d-17b621fb8d28","Type":"ContainerStarted","Data":"6637ecf9558068fb6e0a873a8fc951a3bc2a83bb6006e62ac642f88c6bb9d497"}
Jul 06 22:24:20.253761 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:20.253738    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-105.ec2.internal" event={"ID":"a7e777c6384fb2d7e87381cc5d382585","Type":"ContainerStarted","Data":"1aaa16e2cfa7ce70d9f362ac3c97af22d24b7a1f46829c8eeec8e6af5319e86b"}
Jul 06 22:24:20.253867 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:20.253780    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-z88v4"
Jul 06 22:24:20.254551 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:20.254535    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-z88v4"
Jul 06 22:24:20.288594 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:20.288548    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-105.ec2.internal" podStartSLOduration=25.288529005 podStartE2EDuration="25.288529005s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 22:23:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:24:20.273971298 +0000 UTC m=+25.699017840" watchObservedRunningTime="2026-07-06 22:24:20.288529005 +0000 UTC m=+25.713575545"
Jul 06 22:24:21.257859 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:21.257825    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b8d5j" event={"ID":"e5e3cc00-4650-4700-b30d-17b621fb8d28","Type":"ContainerStarted","Data":"39e70cef4dcb197f3f17d26862071fb73eaf72b39e3b861130c8e8172287d60b"}
Jul 06 22:24:21.276333 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:21.276282    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-b8d5j" podStartSLOduration=4.143363472 podStartE2EDuration="26.276260061s" totalImagesPullingTime="22.132896589s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 22:23:55 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:24:21.276091458 +0000 UTC m=+26.701138001" watchObservedRunningTime="2026-07-06 22:24:21.276260061 +0000 UTC m=+26.701306603"
Jul 06 22:24:22.074111 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:22.074076    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mc48r"
Jul 06 22:24:22.074641 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:22.074080    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6qmpm"
Jul 06 22:24:22.074641 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:22.074080    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bbjgn"
Jul 06 22:24:22.074641 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:24:22.074244    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mc48r" podUID="aad76575-952c-4b85-b64a-2523529c1cf4"
Jul 06 22:24:22.074641 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:24:22.074314    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6qmpm" podUID="da82c66a-1f8a-4315-8f45-8b449e2e46b9"
Jul 06 22:24:22.074641 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:24:22.074412    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bbjgn" podUID="69b0b78e-e934-433b-bfdb-5fd6d8ba4a29"
Jul 06 22:24:23.264079 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:23.263907    2563 generic.go:356] "Generic (PLEG): container finished" podID="9902a376-c450-4867-9c7c-d9e5300bce11" containerID="5393cf385c5ca6141b05f41b0dae87c24a19280a0c47b59b33a64a3826abd67e" exitCode=0
Jul 06 22:24:23.264934 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:23.263993    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rzszm" event={"ID":"9902a376-c450-4867-9c7c-d9e5300bce11","Type":"ContainerDied","Data":"5393cf385c5ca6141b05f41b0dae87c24a19280a0c47b59b33a64a3826abd67e"}
Jul 06 22:24:23.267647 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:23.267631    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-t6jpk_03309f33-ff47-447d-956f-c89bdd4f8054/ovn-acl-logging/0.log"
Jul 06 22:24:23.267978 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:23.267956    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t6jpk" event={"ID":"03309f33-ff47-447d-956f-c89bdd4f8054","Type":"ContainerStarted","Data":"81cf67cda60d0564a46248e7642c2e0e008dd74f05d712a7fbf892bd3d368b50"}
Jul 06 22:24:23.268217 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:23.268201    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-t6jpk"
Jul 06 22:24:23.268351 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:23.268274    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-t6jpk"
Jul 06 22:24:23.268459 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:23.268429    2563 scope.go:117] "RemoveContainer" containerID="ccd5164605b35e284de8e0137cf0cb2053522c0a17944a7da0d972219b5290ec"
Jul 06 22:24:23.286256 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:23.285324    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-t6jpk"
Jul 06 22:24:23.287612 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:23.287580    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-t6jpk"
Jul 06 22:24:24.075057 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:24.074903    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6qmpm"
Jul 06 22:24:24.075271 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:24.074981    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bbjgn"
Jul 06 22:24:24.075271 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:24:24.075138    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6qmpm" podUID="da82c66a-1f8a-4315-8f45-8b449e2e46b9"
Jul 06 22:24:24.075271 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:24.074990    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mc48r"
Jul 06 22:24:24.075271 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:24:24.075237    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bbjgn" podUID="69b0b78e-e934-433b-bfdb-5fd6d8ba4a29"
Jul 06 22:24:24.075421 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:24:24.075283    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mc48r" podUID="aad76575-952c-4b85-b64a-2523529c1cf4"
Jul 06 22:24:24.274382 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:24.274354    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-t6jpk_03309f33-ff47-447d-956f-c89bdd4f8054/ovn-acl-logging/0.log"
Jul 06 22:24:24.274789 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:24.274757    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t6jpk" event={"ID":"03309f33-ff47-447d-956f-c89bdd4f8054","Type":"ContainerStarted","Data":"0074111a2dab542528d4641fc8c7cc32d7e7a1a69c1c5ae29154b89bfd0fc88e"}
Jul 06 22:24:24.274850 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:24.274841    2563 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 06 22:24:24.304243 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:24.304163    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-t6jpk" podStartSLOduration=8.917620863 podStartE2EDuration="29.304147126s" totalImagesPullingTime="20.386526263s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 22:23:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:24:24.302069559 +0000 UTC m=+29.727116099" watchObservedRunningTime="2026-07-06 22:24:24.304147126 +0000 UTC m=+29.729193667"
Jul 06 22:24:24.412446 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:24.412421    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-6qmpm"]
Jul 06 22:24:24.412576 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:24.412540    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6qmpm"
Jul 06 22:24:24.412666 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:24:24.412646    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6qmpm" podUID="da82c66a-1f8a-4315-8f45-8b449e2e46b9"
Jul 06 22:24:24.415730 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:24.415705    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-mc48r"]
Jul 06 22:24:24.415852 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:24.415788    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mc48r"
Jul 06 22:24:24.415932 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:24:24.415906    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mc48r" podUID="aad76575-952c-4b85-b64a-2523529c1cf4"
Jul 06 22:24:24.416506 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:24.416487    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-bbjgn"]
Jul 06 22:24:24.416609 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:24.416551    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bbjgn"
Jul 06 22:24:24.416672 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:24:24.416626    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bbjgn" podUID="69b0b78e-e934-433b-bfdb-5fd6d8ba4a29"
Jul 06 22:24:24.461113 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:24.461085    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-t6jpk"
Jul 06 22:24:25.278583 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:25.278551    2563 generic.go:356] "Generic (PLEG): container finished" podID="9902a376-c450-4867-9c7c-d9e5300bce11" containerID="ca4123748e902acabef8f974b1b930cafb3f18daa2a6db3f75acd921522e6190" exitCode=0
Jul 06 22:24:25.279064 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:25.278636    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rzszm" event={"ID":"9902a376-c450-4867-9c7c-d9e5300bce11","Type":"ContainerDied","Data":"ca4123748e902acabef8f974b1b930cafb3f18daa2a6db3f75acd921522e6190"}
Jul 06 22:24:26.074918 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:26.074837    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mc48r"
Jul 06 22:24:26.074918 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:26.074874    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6qmpm"
Jul 06 22:24:26.075128 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:26.074840    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bbjgn"
Jul 06 22:24:26.075128 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:24:26.074967    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mc48r" podUID="aad76575-952c-4b85-b64a-2523529c1cf4"
Jul 06 22:24:26.075128 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:24:26.075064    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bbjgn" podUID="69b0b78e-e934-433b-bfdb-5fd6d8ba4a29"
Jul 06 22:24:26.075128 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:24:26.075122    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6qmpm" podUID="da82c66a-1f8a-4315-8f45-8b449e2e46b9"
Jul 06 22:24:27.285789 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:27.285757    2563 generic.go:356] "Generic (PLEG): container finished" podID="9902a376-c450-4867-9c7c-d9e5300bce11" containerID="287416ca50ddae9f9244a910b005a64e9faba5cb910eab0bbf27e45e89b4b34a" exitCode=0
Jul 06 22:24:27.286261 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:27.285799    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rzszm" event={"ID":"9902a376-c450-4867-9c7c-d9e5300bce11","Type":"ContainerDied","Data":"287416ca50ddae9f9244a910b005a64e9faba5cb910eab0bbf27e45e89b4b34a"}
Jul 06 22:24:27.739013 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:27.738964    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gvmwj\" (UniqueName: \"kubernetes.io/projected/da82c66a-1f8a-4315-8f45-8b449e2e46b9-kube-api-access-gvmwj\") pod \"network-check-target-6qmpm\" (UID: \"da82c66a-1f8a-4315-8f45-8b449e2e46b9\") " pod="openshift-network-diagnostics/network-check-target-6qmpm"
Jul 06 22:24:27.739230 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:24:27.739142    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 06 22:24:27.739230 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:24:27.739185    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 06 22:24:27.739230 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:24:27.739200    2563 projected.go:196] Error preparing data for projected volume kube-api-access-gvmwj for pod openshift-network-diagnostics/network-check-target-6qmpm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 22:24:27.739405 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:24:27.739260    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/da82c66a-1f8a-4315-8f45-8b449e2e46b9-kube-api-access-gvmwj podName:da82c66a-1f8a-4315-8f45-8b449e2e46b9 nodeName:}" failed. No retries permitted until 2026-07-06 22:24:59.739243414 +0000 UTC m=+65.164289939 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-gvmwj" (UniqueName: "kubernetes.io/projected/da82c66a-1f8a-4315-8f45-8b449e2e46b9-kube-api-access-gvmwj") pod "network-check-target-6qmpm" (UID: "da82c66a-1f8a-4315-8f45-8b449e2e46b9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 22:24:27.839461 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:27.839431    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/aad76575-952c-4b85-b64a-2523529c1cf4-original-pull-secret\") pod \"global-pull-secret-syncer-mc48r\" (UID: \"aad76575-952c-4b85-b64a-2523529c1cf4\") " pod="kube-system/global-pull-secret-syncer-mc48r"
Jul 06 22:24:27.839628 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:27.839494    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/69b0b78e-e934-433b-bfdb-5fd6d8ba4a29-metrics-certs\") pod \"network-metrics-daemon-bbjgn\" (UID: \"69b0b78e-e934-433b-bfdb-5fd6d8ba4a29\") " pod="openshift-multus/network-metrics-daemon-bbjgn"
Jul 06 22:24:27.839628 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:24:27.839567    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 06 22:24:27.839628 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:24:27.839585    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 22:24:27.839765 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:24:27.839635    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aad76575-952c-4b85-b64a-2523529c1cf4-original-pull-secret podName:aad76575-952c-4b85-b64a-2523529c1cf4 nodeName:}" failed. No retries permitted until 2026-07-06 22:24:59.839618141 +0000 UTC m=+65.264664673 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/aad76575-952c-4b85-b64a-2523529c1cf4-original-pull-secret") pod "global-pull-secret-syncer-mc48r" (UID: "aad76575-952c-4b85-b64a-2523529c1cf4") : object "kube-system"/"original-pull-secret" not registered
Jul 06 22:24:27.839765 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:24:27.839652    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/69b0b78e-e934-433b-bfdb-5fd6d8ba4a29-metrics-certs podName:69b0b78e-e934-433b-bfdb-5fd6d8ba4a29 nodeName:}" failed. No retries permitted until 2026-07-06 22:24:59.839643664 +0000 UTC m=+65.264690183 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/69b0b78e-e934-433b-bfdb-5fd6d8ba4a29-metrics-certs") pod "network-metrics-daemon-bbjgn" (UID: "69b0b78e-e934-433b-bfdb-5fd6d8ba4a29") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 22:24:28.075035 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:28.074736    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6qmpm"
Jul 06 22:24:28.075199 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:28.074749    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mc48r"
Jul 06 22:24:28.075199 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:24:28.075081    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6qmpm" podUID="da82c66a-1f8a-4315-8f45-8b449e2e46b9"
Jul 06 22:24:28.075199 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:28.074749    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bbjgn"
Jul 06 22:24:28.075199 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:24:28.075146    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mc48r" podUID="aad76575-952c-4b85-b64a-2523529c1cf4"
Jul 06 22:24:28.075356 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:24:28.075266    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bbjgn" podUID="69b0b78e-e934-433b-bfdb-5fd6d8ba4a29"
Jul 06 22:24:29.898367 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:29.898335    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-105.ec2.internal" event="NodeReady"
Jul 06 22:24:29.898848 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:29.898483    2563 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 06 22:24:29.946108 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:29.946077    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-bv5xq"]
Jul 06 22:24:29.968733 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:29.968695    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-78psz"]
Jul 06 22:24:29.968885 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:29.968837    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-bv5xq"
Jul 06 22:24:29.971391 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:29.971357    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 06 22:24:29.971391 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:29.971381    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 06 22:24:29.971544 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:29.971513    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-2cqxg\""
Jul 06 22:24:29.988266 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:29.988231    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-84f89db6f7-vbq96"]
Jul 06 22:24:29.988606 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:29.988577    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-78psz"
Jul 06 22:24:29.991292 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:29.991270    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 06 22:24:29.991578 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:29.991553    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 06 22:24:29.991940 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:29.991927    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 06 22:24:29.992075 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:29.992064    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-tzl8n\""
Jul 06 22:24:30.012030 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:30.012006    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-78psz"]
Jul 06 22:24:30.012223 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:30.012205    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-bv5xq"]
Jul 06 22:24:30.012343 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:30.012128    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-84f89db6f7-vbq96"
Jul 06 22:24:30.012343 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:30.012236    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-84f89db6f7-vbq96"]
Jul 06 22:24:30.014686 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:30.014666    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 06 22:24:30.014686 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:30.014678    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 06 22:24:30.014855 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:30.014768    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-c4gjv\""
Jul 06 22:24:30.015147 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:30.015130    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 06 22:24:30.020524 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:30.020506    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 06 22:24:30.054116 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:30.054086    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/62bbe928-f2d0-4642-a2a3-f272341820eb-config-volume\") pod \"dns-default-bv5xq\" (UID: \"62bbe928-f2d0-4642-a2a3-f272341820eb\") " pod="openshift-dns/dns-default-bv5xq"
Jul 06 22:24:30.054273 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:30.054138    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/62bbe928-f2d0-4642-a2a3-f272341820eb-metrics-tls\") pod \"dns-default-bv5xq\" (UID: \"62bbe928-f2d0-4642-a2a3-f272341820eb\") " pod="openshift-dns/dns-default-bv5xq"
Jul 06 22:24:30.054273 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:30.054217    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6e143322-bd8b-42e9-ab2b-7071044779a8-cert\") pod \"ingress-canary-78psz\" (UID: \"6e143322-bd8b-42e9-ab2b-7071044779a8\") " pod="openshift-ingress-canary/ingress-canary-78psz"
Jul 06 22:24:30.054397 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:30.054293    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5m82c\" (UniqueName: \"kubernetes.io/projected/6e143322-bd8b-42e9-ab2b-7071044779a8-kube-api-access-5m82c\") pod \"ingress-canary-78psz\" (UID: \"6e143322-bd8b-42e9-ab2b-7071044779a8\") " pod="openshift-ingress-canary/ingress-canary-78psz"
Jul 06 22:24:30.054397 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:30.054335    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z5mh4\" (UniqueName: \"kubernetes.io/projected/62bbe928-f2d0-4642-a2a3-f272341820eb-kube-api-access-z5mh4\") pod \"dns-default-bv5xq\" (UID: \"62bbe928-f2d0-4642-a2a3-f272341820eb\") " pod="openshift-dns/dns-default-bv5xq"
Jul 06 22:24:30.054397 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:30.054363    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/62bbe928-f2d0-4642-a2a3-f272341820eb-tmp-dir\") pod \"dns-default-bv5xq\" (UID: \"62bbe928-f2d0-4642-a2a3-f272341820eb\") " pod="openshift-dns/dns-default-bv5xq"
Jul 06 22:24:30.074382 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:30.074349    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6qmpm"
Jul 06 22:24:30.074382 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:30.074368    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bbjgn"
Jul 06 22:24:30.074566 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:30.074464    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mc48r"
Jul 06 22:24:30.077141 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:30.077120    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 06 22:24:30.077141 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:30.077135    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 06 22:24:30.077336 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:30.077253    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 06 22:24:30.077523 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:30.077508    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-zvxv7\""
Jul 06 22:24:30.077592 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:30.077530    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-zhlqr\""
Jul 06 22:24:30.077592 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:30.077542    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 06 22:24:30.155784 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:30.155708    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/63e27075-6d1f-43a1-82bf-ca8e43da8326-registry-tls\") pod \"image-registry-84f89db6f7-vbq96\" (UID: \"63e27075-6d1f-43a1-82bf-ca8e43da8326\") " pod="openshift-image-registry/image-registry-84f89db6f7-vbq96"
Jul 06 22:24:30.155784 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:30.155757    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/63e27075-6d1f-43a1-82bf-ca8e43da8326-trusted-ca\") pod \"image-registry-84f89db6f7-vbq96\" (UID: \"63e27075-6d1f-43a1-82bf-ca8e43da8326\") " pod="openshift-image-registry/image-registry-84f89db6f7-vbq96"
Jul 06 22:24:30.156012 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:30.155832    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/62bbe928-f2d0-4642-a2a3-f272341820eb-metrics-tls\") pod \"dns-default-bv5xq\" (UID: \"62bbe928-f2d0-4642-a2a3-f272341820eb\") " pod="openshift-dns/dns-default-bv5xq"
Jul 06 22:24:30.156012 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:30.155876    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6e143322-bd8b-42e9-ab2b-7071044779a8-cert\") pod \"ingress-canary-78psz\" (UID: \"6e143322-bd8b-42e9-ab2b-7071044779a8\") " pod="openshift-ingress-canary/ingress-canary-78psz"
Jul 06 22:24:30.156012 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:24:30.155980    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 06 22:24:30.156155 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:24:30.156085    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/62bbe928-f2d0-4642-a2a3-f272341820eb-metrics-tls podName:62bbe928-f2d0-4642-a2a3-f272341820eb nodeName:}" failed. No retries permitted until 2026-07-06 22:24:30.65606088 +0000 UTC m=+36.081107403 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/62bbe928-f2d0-4642-a2a3-f272341820eb-metrics-tls") pod "dns-default-bv5xq" (UID: "62bbe928-f2d0-4642-a2a3-f272341820eb") : secret "dns-default-metrics-tls" not found
Jul 06 22:24:30.156249 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:30.156232    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5m82c\" (UniqueName: \"kubernetes.io/projected/6e143322-bd8b-42e9-ab2b-7071044779a8-kube-api-access-5m82c\") pod \"ingress-canary-78psz\" (UID: \"6e143322-bd8b-42e9-ab2b-7071044779a8\") " pod="openshift-ingress-canary/ingress-canary-78psz"
Jul 06 22:24:30.156297 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:30.156264    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/63e27075-6d1f-43a1-82bf-ca8e43da8326-registry-certificates\") pod \"image-registry-84f89db6f7-vbq96\" (UID: \"63e27075-6d1f-43a1-82bf-ca8e43da8326\") " pod="openshift-image-registry/image-registry-84f89db6f7-vbq96"
Jul 06 22:24:30.156347 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:30.156301    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z5mh4\" (UniqueName: \"kubernetes.io/projected/62bbe928-f2d0-4642-a2a3-f272341820eb-kube-api-access-z5mh4\") pod \"dns-default-bv5xq\" (UID: \"62bbe928-f2d0-4642-a2a3-f272341820eb\") " pod="openshift-dns/dns-default-bv5xq"
Jul 06 22:24:30.156347 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:30.156326    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/63e27075-6d1f-43a1-82bf-ca8e43da8326-ca-trust-extracted\") pod \"image-registry-84f89db6f7-vbq96\" (UID: \"63e27075-6d1f-43a1-82bf-ca8e43da8326\") " pod="openshift-image-registry/image-registry-84f89db6f7-vbq96"
Jul 06 22:24:30.156442 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:30.156370    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/63e27075-6d1f-43a1-82bf-ca8e43da8326-bound-sa-token\") pod \"image-registry-84f89db6f7-vbq96\" (UID: \"63e27075-6d1f-43a1-82bf-ca8e43da8326\") " pod="openshift-image-registry/image-registry-84f89db6f7-vbq96"
Jul 06 22:24:30.156442 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:30.156408    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/62bbe928-f2d0-4642-a2a3-f272341820eb-tmp-dir\") pod \"dns-default-bv5xq\" (UID: \"62bbe928-f2d0-4642-a2a3-f272341820eb\") " pod="openshift-dns/dns-default-bv5xq"
Jul 06 22:24:30.156531 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:30.156440    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k6sd7\" (UniqueName: \"kubernetes.io/projected/63e27075-6d1f-43a1-82bf-ca8e43da8326-kube-api-access-k6sd7\") pod \"image-registry-84f89db6f7-vbq96\" (UID: \"63e27075-6d1f-43a1-82bf-ca8e43da8326\") " pod="openshift-image-registry/image-registry-84f89db6f7-vbq96"
Jul 06 22:24:30.156531 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:30.156483    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/62bbe928-f2d0-4642-a2a3-f272341820eb-config-volume\") pod \"dns-default-bv5xq\" (UID: \"62bbe928-f2d0-4642-a2a3-f272341820eb\") " pod="openshift-dns/dns-default-bv5xq"
Jul 06 22:24:30.156623 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:30.156557    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/63e27075-6d1f-43a1-82bf-ca8e43da8326-installation-pull-secrets\") pod \"image-registry-84f89db6f7-vbq96\" (UID: \"63e27075-6d1f-43a1-82bf-ca8e43da8326\") " pod="openshift-image-registry/image-registry-84f89db6f7-vbq96"
Jul 06 22:24:30.156714 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:30.156693    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/63e27075-6d1f-43a1-82bf-ca8e43da8326-image-registry-private-configuration\") pod \"image-registry-84f89db6f7-vbq96\" (UID: \"63e27075-6d1f-43a1-82bf-ca8e43da8326\") " pod="openshift-image-registry/image-registry-84f89db6f7-vbq96"
Jul 06 22:24:30.156786 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:30.156740    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/62bbe928-f2d0-4642-a2a3-f272341820eb-tmp-dir\") pod \"dns-default-bv5xq\" (UID: \"62bbe928-f2d0-4642-a2a3-f272341820eb\") " pod="openshift-dns/dns-default-bv5xq"
Jul 06 22:24:30.157051 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:30.157031    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/62bbe928-f2d0-4642-a2a3-f272341820eb-config-volume\") pod \"dns-default-bv5xq\" (UID: \"62bbe928-f2d0-4642-a2a3-f272341820eb\") " pod="openshift-dns/dns-default-bv5xq"
Jul 06 22:24:30.166665 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:30.165799    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6e143322-bd8b-42e9-ab2b-7071044779a8-cert\") pod \"ingress-canary-78psz\" (UID: \"6e143322-bd8b-42e9-ab2b-7071044779a8\") " pod="openshift-ingress-canary/ingress-canary-78psz"
Jul 06 22:24:30.166665 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:30.165864    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5m82c\" (UniqueName: \"kubernetes.io/projected/6e143322-bd8b-42e9-ab2b-7071044779a8-kube-api-access-5m82c\") pod \"ingress-canary-78psz\" (UID: \"6e143322-bd8b-42e9-ab2b-7071044779a8\") " pod="openshift-ingress-canary/ingress-canary-78psz"
Jul 06 22:24:30.166665 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:30.165983    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z5mh4\" (UniqueName: \"kubernetes.io/projected/62bbe928-f2d0-4642-a2a3-f272341820eb-kube-api-access-z5mh4\") pod \"dns-default-bv5xq\" (UID: \"62bbe928-f2d0-4642-a2a3-f272341820eb\") " pod="openshift-dns/dns-default-bv5xq"
Jul 06 22:24:30.257278 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:30.257241    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/63e27075-6d1f-43a1-82bf-ca8e43da8326-installation-pull-secrets\") pod \"image-registry-84f89db6f7-vbq96\" (UID: \"63e27075-6d1f-43a1-82bf-ca8e43da8326\") " pod="openshift-image-registry/image-registry-84f89db6f7-vbq96"
Jul 06 22:24:30.257449 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:30.257289    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/63e27075-6d1f-43a1-82bf-ca8e43da8326-image-registry-private-configuration\") pod \"image-registry-84f89db6f7-vbq96\" (UID: \"63e27075-6d1f-43a1-82bf-ca8e43da8326\") " pod="openshift-image-registry/image-registry-84f89db6f7-vbq96"
Jul 06 22:24:30.257449 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:30.257312    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/63e27075-6d1f-43a1-82bf-ca8e43da8326-registry-tls\") pod \"image-registry-84f89db6f7-vbq96\" (UID: \"63e27075-6d1f-43a1-82bf-ca8e43da8326\") " pod="openshift-image-registry/image-registry-84f89db6f7-vbq96"
Jul 06 22:24:30.257449 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:30.257342    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/63e27075-6d1f-43a1-82bf-ca8e43da8326-trusted-ca\") pod \"image-registry-84f89db6f7-vbq96\" (UID: \"63e27075-6d1f-43a1-82bf-ca8e43da8326\") " pod="openshift-image-registry/image-registry-84f89db6f7-vbq96"
Jul 06 22:24:30.257449 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:30.257415    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/63e27075-6d1f-43a1-82bf-ca8e43da8326-registry-certificates\") pod \"image-registry-84f89db6f7-vbq96\" (UID: \"63e27075-6d1f-43a1-82bf-ca8e43da8326\") " pod="openshift-image-registry/image-registry-84f89db6f7-vbq96"
Jul 06 22:24:30.257739 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:30.257457    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/63e27075-6d1f-43a1-82bf-ca8e43da8326-ca-trust-extracted\") pod \"image-registry-84f89db6f7-vbq96\" (UID: \"63e27075-6d1f-43a1-82bf-ca8e43da8326\") " pod="openshift-image-registry/image-registry-84f89db6f7-vbq96"
Jul 06 22:24:30.257739 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:30.257479    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/63e27075-6d1f-43a1-82bf-ca8e43da8326-bound-sa-token\") pod \"image-registry-84f89db6f7-vbq96\" (UID: \"63e27075-6d1f-43a1-82bf-ca8e43da8326\") " pod="openshift-image-registry/image-registry-84f89db6f7-vbq96"
Jul 06 22:24:30.257739 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:24:30.257491    2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 06 22:24:30.257739 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:24:30.257507    2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-84f89db6f7-vbq96: secret "image-registry-tls" not found
Jul 06 22:24:30.257739 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:30.257511    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k6sd7\" (UniqueName: \"kubernetes.io/projected/63e27075-6d1f-43a1-82bf-ca8e43da8326-kube-api-access-k6sd7\") pod \"image-registry-84f89db6f7-vbq96\" (UID: \"63e27075-6d1f-43a1-82bf-ca8e43da8326\") " pod="openshift-image-registry/image-registry-84f89db6f7-vbq96"
Jul 06 22:24:30.257739 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:24:30.257579    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/63e27075-6d1f-43a1-82bf-ca8e43da8326-registry-tls podName:63e27075-6d1f-43a1-82bf-ca8e43da8326 nodeName:}" failed. No retries permitted until 2026-07-06 22:24:30.757556696 +0000 UTC m=+36.182603237 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/63e27075-6d1f-43a1-82bf-ca8e43da8326-registry-tls") pod "image-registry-84f89db6f7-vbq96" (UID: "63e27075-6d1f-43a1-82bf-ca8e43da8326") : secret "image-registry-tls" not found
Jul 06 22:24:30.258030 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:30.258007    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/63e27075-6d1f-43a1-82bf-ca8e43da8326-ca-trust-extracted\") pod \"image-registry-84f89db6f7-vbq96\" (UID: \"63e27075-6d1f-43a1-82bf-ca8e43da8326\") " pod="openshift-image-registry/image-registry-84f89db6f7-vbq96"
Jul 06 22:24:30.258589 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:30.258559    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/63e27075-6d1f-43a1-82bf-ca8e43da8326-registry-certificates\") pod \"image-registry-84f89db6f7-vbq96\" (UID: \"63e27075-6d1f-43a1-82bf-ca8e43da8326\") " pod="openshift-image-registry/image-registry-84f89db6f7-vbq96"
Jul 06 22:24:30.258744 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:30.258722    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/63e27075-6d1f-43a1-82bf-ca8e43da8326-trusted-ca\") pod \"image-registry-84f89db6f7-vbq96\" (UID: \"63e27075-6d1f-43a1-82bf-ca8e43da8326\") " pod="openshift-image-registry/image-registry-84f89db6f7-vbq96"
Jul 06 22:24:30.260017 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:30.259993    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/63e27075-6d1f-43a1-82bf-ca8e43da8326-image-registry-private-configuration\") pod \"image-registry-84f89db6f7-vbq96\" (UID: \"63e27075-6d1f-43a1-82bf-ca8e43da8326\") " pod="openshift-image-registry/image-registry-84f89db6f7-vbq96"
Jul 06 22:24:30.260332 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:30.260309    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/63e27075-6d1f-43a1-82bf-ca8e43da8326-installation-pull-secrets\") pod \"image-registry-84f89db6f7-vbq96\" (UID: \"63e27075-6d1f-43a1-82bf-ca8e43da8326\") " pod="openshift-image-registry/image-registry-84f89db6f7-vbq96"
Jul 06 22:24:30.267516 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:30.267497    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k6sd7\" (UniqueName: \"kubernetes.io/projected/63e27075-6d1f-43a1-82bf-ca8e43da8326-kube-api-access-k6sd7\") pod \"image-registry-84f89db6f7-vbq96\" (UID: \"63e27075-6d1f-43a1-82bf-ca8e43da8326\") " pod="openshift-image-registry/image-registry-84f89db6f7-vbq96"
Jul 06 22:24:30.267599 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:30.267571    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/63e27075-6d1f-43a1-82bf-ca8e43da8326-bound-sa-token\") pod \"image-registry-84f89db6f7-vbq96\" (UID: \"63e27075-6d1f-43a1-82bf-ca8e43da8326\") " pod="openshift-image-registry/image-registry-84f89db6f7-vbq96"
Jul 06 22:24:30.299221 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:30.299193    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-78psz"
Jul 06 22:24:30.660771 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:30.660741    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/62bbe928-f2d0-4642-a2a3-f272341820eb-metrics-tls\") pod \"dns-default-bv5xq\" (UID: \"62bbe928-f2d0-4642-a2a3-f272341820eb\") " pod="openshift-dns/dns-default-bv5xq"
Jul 06 22:24:30.660940 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:24:30.660915    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 06 22:24:30.660989 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:24:30.660981    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/62bbe928-f2d0-4642-a2a3-f272341820eb-metrics-tls podName:62bbe928-f2d0-4642-a2a3-f272341820eb nodeName:}" failed. No retries permitted until 2026-07-06 22:24:31.660965028 +0000 UTC m=+37.086011551 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/62bbe928-f2d0-4642-a2a3-f272341820eb-metrics-tls") pod "dns-default-bv5xq" (UID: "62bbe928-f2d0-4642-a2a3-f272341820eb") : secret "dns-default-metrics-tls" not found
Jul 06 22:24:30.761416 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:30.761380    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/63e27075-6d1f-43a1-82bf-ca8e43da8326-registry-tls\") pod \"image-registry-84f89db6f7-vbq96\" (UID: \"63e27075-6d1f-43a1-82bf-ca8e43da8326\") " pod="openshift-image-registry/image-registry-84f89db6f7-vbq96"
Jul 06 22:24:30.761581 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:24:30.761531    2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 06 22:24:30.761581 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:24:30.761554    2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-84f89db6f7-vbq96: secret "image-registry-tls" not found
Jul 06 22:24:30.761698 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:24:30.761619    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/63e27075-6d1f-43a1-82bf-ca8e43da8326-registry-tls podName:63e27075-6d1f-43a1-82bf-ca8e43da8326 nodeName:}" failed. No retries permitted until 2026-07-06 22:24:31.761600294 +0000 UTC m=+37.186646832 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/63e27075-6d1f-43a1-82bf-ca8e43da8326-registry-tls") pod "image-registry-84f89db6f7-vbq96" (UID: "63e27075-6d1f-43a1-82bf-ca8e43da8326") : secret "image-registry-tls" not found
Jul 06 22:24:31.596024 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:31.595992    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-78psz"]
Jul 06 22:24:31.600008 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:24:31.599984    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e143322_bd8b_42e9_ab2b_7071044779a8.slice/crio-11c156d5daf7ecba45a53d5d96b42cd4f66c39c9d807e9ad215349175b8b7398 WatchSource:0}: Error finding container 11c156d5daf7ecba45a53d5d96b42cd4f66c39c9d807e9ad215349175b8b7398: Status 404 returned error can't find the container with id 11c156d5daf7ecba45a53d5d96b42cd4f66c39c9d807e9ad215349175b8b7398
Jul 06 22:24:31.672692 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:31.672490    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/62bbe928-f2d0-4642-a2a3-f272341820eb-metrics-tls\") pod \"dns-default-bv5xq\" (UID: \"62bbe928-f2d0-4642-a2a3-f272341820eb\") " pod="openshift-dns/dns-default-bv5xq"
Jul 06 22:24:31.672806 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:24:31.672638    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 06 22:24:31.672806 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:24:31.672744    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/62bbe928-f2d0-4642-a2a3-f272341820eb-metrics-tls podName:62bbe928-f2d0-4642-a2a3-f272341820eb nodeName:}" failed. No retries permitted until 2026-07-06 22:24:33.672730602 +0000 UTC m=+39.097777122 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/62bbe928-f2d0-4642-a2a3-f272341820eb-metrics-tls") pod "dns-default-bv5xq" (UID: "62bbe928-f2d0-4642-a2a3-f272341820eb") : secret "dns-default-metrics-tls" not found
Jul 06 22:24:31.773461 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:31.773439    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/63e27075-6d1f-43a1-82bf-ca8e43da8326-registry-tls\") pod \"image-registry-84f89db6f7-vbq96\" (UID: \"63e27075-6d1f-43a1-82bf-ca8e43da8326\") " pod="openshift-image-registry/image-registry-84f89db6f7-vbq96"
Jul 06 22:24:31.773588 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:24:31.773574    2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 06 22:24:31.773635 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:24:31.773590    2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-84f89db6f7-vbq96: secret "image-registry-tls" not found
Jul 06 22:24:31.773669 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:24:31.773638    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/63e27075-6d1f-43a1-82bf-ca8e43da8326-registry-tls podName:63e27075-6d1f-43a1-82bf-ca8e43da8326 nodeName:}" failed. No retries permitted until 2026-07-06 22:24:33.773622943 +0000 UTC m=+39.198669462 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/63e27075-6d1f-43a1-82bf-ca8e43da8326-registry-tls") pod "image-registry-84f89db6f7-vbq96" (UID: "63e27075-6d1f-43a1-82bf-ca8e43da8326") : secret "image-registry-tls" not found
Jul 06 22:24:32.299940 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:32.299872    2563 generic.go:356] "Generic (PLEG): container finished" podID="9902a376-c450-4867-9c7c-d9e5300bce11" containerID="5986dcca9ef4d7476044f1288eed01bf035806843e63b051d84e26c7743a0740" exitCode=0
Jul 06 22:24:32.300079 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:32.299961    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rzszm" event={"ID":"9902a376-c450-4867-9c7c-d9e5300bce11","Type":"ContainerDied","Data":"5986dcca9ef4d7476044f1288eed01bf035806843e63b051d84e26c7743a0740"}
Jul 06 22:24:32.301430 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:32.301401    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-78psz" event={"ID":"6e143322-bd8b-42e9-ab2b-7071044779a8","Type":"ContainerStarted","Data":"11c156d5daf7ecba45a53d5d96b42cd4f66c39c9d807e9ad215349175b8b7398"}
Jul 06 22:24:33.307350 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:33.307265    2563 generic.go:356] "Generic (PLEG): container finished" podID="9902a376-c450-4867-9c7c-d9e5300bce11" containerID="f70b18305989c327153b83c547471e2e2cdc7cd1826c217a5f56044a5d0eb85f" exitCode=0
Jul 06 22:24:33.307762 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:33.307350    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rzszm" event={"ID":"9902a376-c450-4867-9c7c-d9e5300bce11","Type":"ContainerDied","Data":"f70b18305989c327153b83c547471e2e2cdc7cd1826c217a5f56044a5d0eb85f"}
Jul 06 22:24:33.686666 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:33.686573    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/62bbe928-f2d0-4642-a2a3-f272341820eb-metrics-tls\") pod \"dns-default-bv5xq\" (UID: \"62bbe928-f2d0-4642-a2a3-f272341820eb\") " pod="openshift-dns/dns-default-bv5xq"
Jul 06 22:24:33.686812 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:24:33.686742    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 06 22:24:33.686812 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:24:33.686811    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/62bbe928-f2d0-4642-a2a3-f272341820eb-metrics-tls podName:62bbe928-f2d0-4642-a2a3-f272341820eb nodeName:}" failed. No retries permitted until 2026-07-06 22:24:37.686793072 +0000 UTC m=+43.111839595 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/62bbe928-f2d0-4642-a2a3-f272341820eb-metrics-tls") pod "dns-default-bv5xq" (UID: "62bbe928-f2d0-4642-a2a3-f272341820eb") : secret "dns-default-metrics-tls" not found
Jul 06 22:24:33.787698 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:33.787678    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/63e27075-6d1f-43a1-82bf-ca8e43da8326-registry-tls\") pod \"image-registry-84f89db6f7-vbq96\" (UID: \"63e27075-6d1f-43a1-82bf-ca8e43da8326\") " pod="openshift-image-registry/image-registry-84f89db6f7-vbq96"
Jul 06 22:24:33.787787 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:24:33.787776    2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 06 22:24:33.787833 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:24:33.787790    2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-84f89db6f7-vbq96: secret "image-registry-tls" not found
Jul 06 22:24:33.787833 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:24:33.787832    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/63e27075-6d1f-43a1-82bf-ca8e43da8326-registry-tls podName:63e27075-6d1f-43a1-82bf-ca8e43da8326 nodeName:}" failed. No retries permitted until 2026-07-06 22:24:37.787819235 +0000 UTC m=+43.212865758 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/63e27075-6d1f-43a1-82bf-ca8e43da8326-registry-tls") pod "image-registry-84f89db6f7-vbq96" (UID: "63e27075-6d1f-43a1-82bf-ca8e43da8326") : secret "image-registry-tls" not found
Jul 06 22:24:34.313362 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:34.313328    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rzszm" event={"ID":"9902a376-c450-4867-9c7c-d9e5300bce11","Type":"ContainerStarted","Data":"1405b9277afc66b10d3b4589a3206504caed3335e555b05e15613088dceef7ff"}
Jul 06 22:24:34.341271 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:34.341227    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-rzszm" podStartSLOduration=8.335237064 podStartE2EDuration="39.341211258s" totalImagesPullingTime="31.005974194s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 22:23:55 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:24:34.339808278 +0000 UTC m=+39.764854820" watchObservedRunningTime="2026-07-06 22:24:34.341211258 +0000 UTC m=+39.766257800"
Jul 06 22:24:35.317035 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:35.316998    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-78psz" event={"ID":"6e143322-bd8b-42e9-ab2b-7071044779a8","Type":"ContainerStarted","Data":"d1f5f79e055e5389b3959917c7ee28e9d5ddbee92bcea1353b8da087b1ce55e5"}
Jul 06 22:24:35.332949 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:35.332894    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-78psz" podStartSLOduration=3.6620923420000002 podStartE2EDuration="6.332877978s" totalImagesPullingTime="2.670785636s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:24:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:24:35.332244313 +0000 UTC m=+40.757290853" watchObservedRunningTime="2026-07-06 22:24:35.332877978 +0000 UTC m=+40.757924518"
Jul 06 22:24:37.717145 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:37.717106    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/62bbe928-f2d0-4642-a2a3-f272341820eb-metrics-tls\") pod \"dns-default-bv5xq\" (UID: \"62bbe928-f2d0-4642-a2a3-f272341820eb\") " pod="openshift-dns/dns-default-bv5xq"
Jul 06 22:24:37.717649 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:24:37.717289    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 06 22:24:37.717649 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:24:37.717370    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/62bbe928-f2d0-4642-a2a3-f272341820eb-metrics-tls podName:62bbe928-f2d0-4642-a2a3-f272341820eb nodeName:}" failed. No retries permitted until 2026-07-06 22:24:45.717350937 +0000 UTC m=+51.142397457 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/62bbe928-f2d0-4642-a2a3-f272341820eb-metrics-tls") pod "dns-default-bv5xq" (UID: "62bbe928-f2d0-4642-a2a3-f272341820eb") : secret "dns-default-metrics-tls" not found
Jul 06 22:24:37.818157 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:37.818112    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/63e27075-6d1f-43a1-82bf-ca8e43da8326-registry-tls\") pod \"image-registry-84f89db6f7-vbq96\" (UID: \"63e27075-6d1f-43a1-82bf-ca8e43da8326\") " pod="openshift-image-registry/image-registry-84f89db6f7-vbq96"
Jul 06 22:24:37.818306 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:24:37.818269    2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 06 22:24:37.818306 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:24:37.818287    2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-84f89db6f7-vbq96: secret "image-registry-tls" not found
Jul 06 22:24:37.818398 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:24:37.818339    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/63e27075-6d1f-43a1-82bf-ca8e43da8326-registry-tls podName:63e27075-6d1f-43a1-82bf-ca8e43da8326 nodeName:}" failed. No retries permitted until 2026-07-06 22:24:45.818324171 +0000 UTC m=+51.243370690 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/63e27075-6d1f-43a1-82bf-ca8e43da8326-registry-tls") pod "image-registry-84f89db6f7-vbq96" (UID: "63e27075-6d1f-43a1-82bf-ca8e43da8326") : secret "image-registry-tls" not found
Jul 06 22:24:39.032839 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:39.031212    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-jxjpn"]
Jul 06 22:24:39.034130 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:39.034103    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-jxjpn"
Jul 06 22:24:39.037089 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:39.037058    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 06 22:24:39.037089 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:39.037083    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 06 22:24:39.037237 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:39.037124    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-9tvqz\""
Jul 06 22:24:39.038036 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:39.038016    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 06 22:24:39.038136 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:39.038040    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 06 22:24:39.050019 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:39.049998    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-jxjpn"]
Jul 06 22:24:39.126951 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:39.126917    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/5e411fd6-9619-4fb9-a556-f20259e121e0-crio-socket\") pod \"insights-runtime-extractor-jxjpn\" (UID: \"5e411fd6-9619-4fb9-a556-f20259e121e0\") " pod="openshift-insights/insights-runtime-extractor-jxjpn"
Jul 06 22:24:39.127079 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:39.126975    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5e411fd6-9619-4fb9-a556-f20259e121e0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jxjpn\" (UID: \"5e411fd6-9619-4fb9-a556-f20259e121e0\") " pod="openshift-insights/insights-runtime-extractor-jxjpn"
Jul 06 22:24:39.127079 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:39.126993    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5e411fd6-9619-4fb9-a556-f20259e121e0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jxjpn\" (UID: \"5e411fd6-9619-4fb9-a556-f20259e121e0\") " pod="openshift-insights/insights-runtime-extractor-jxjpn"
Jul 06 22:24:39.127079 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:39.127020    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sd288\" (UniqueName: \"kubernetes.io/projected/5e411fd6-9619-4fb9-a556-f20259e121e0-kube-api-access-sd288\") pod \"insights-runtime-extractor-jxjpn\" (UID: \"5e411fd6-9619-4fb9-a556-f20259e121e0\") " pod="openshift-insights/insights-runtime-extractor-jxjpn"
Jul 06 22:24:39.127079 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:39.127045    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/5e411fd6-9619-4fb9-a556-f20259e121e0-data-volume\") pod \"insights-runtime-extractor-jxjpn\" (UID: \"5e411fd6-9619-4fb9-a556-f20259e121e0\") " pod="openshift-insights/insights-runtime-extractor-jxjpn"
Jul 06 22:24:39.228291 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:39.228257    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/5e411fd6-9619-4fb9-a556-f20259e121e0-crio-socket\") pod \"insights-runtime-extractor-jxjpn\" (UID: \"5e411fd6-9619-4fb9-a556-f20259e121e0\") " pod="openshift-insights/insights-runtime-extractor-jxjpn"
Jul 06 22:24:39.228413 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:39.228307    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5e411fd6-9619-4fb9-a556-f20259e121e0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jxjpn\" (UID: \"5e411fd6-9619-4fb9-a556-f20259e121e0\") " pod="openshift-insights/insights-runtime-extractor-jxjpn"
Jul 06 22:24:39.228413 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:39.228325    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5e411fd6-9619-4fb9-a556-f20259e121e0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jxjpn\" (UID: \"5e411fd6-9619-4fb9-a556-f20259e121e0\") " pod="openshift-insights/insights-runtime-extractor-jxjpn"
Jul 06 22:24:39.228413 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:39.228352    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sd288\" (UniqueName: \"kubernetes.io/projected/5e411fd6-9619-4fb9-a556-f20259e121e0-kube-api-access-sd288\") pod \"insights-runtime-extractor-jxjpn\" (UID: \"5e411fd6-9619-4fb9-a556-f20259e121e0\") " pod="openshift-insights/insights-runtime-extractor-jxjpn"
Jul 06 22:24:39.228413 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:39.228380    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/5e411fd6-9619-4fb9-a556-f20259e121e0-data-volume\") pod \"insights-runtime-extractor-jxjpn\" (UID: \"5e411fd6-9619-4fb9-a556-f20259e121e0\") " pod="openshift-insights/insights-runtime-extractor-jxjpn"
Jul 06 22:24:39.228413 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:24:39.228399    2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 06 22:24:39.228621 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:24:39.228474    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5e411fd6-9619-4fb9-a556-f20259e121e0-kube-rbac-proxy-cm podName:5e411fd6-9619-4fb9-a556-f20259e121e0 nodeName:}" failed. No retries permitted until 2026-07-06 22:24:39.728453894 +0000 UTC m=+45.153500452 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/5e411fd6-9619-4fb9-a556-f20259e121e0-kube-rbac-proxy-cm") pod "insights-runtime-extractor-jxjpn" (UID: "5e411fd6-9619-4fb9-a556-f20259e121e0") : configmap "kube-rbac-proxy" not found
Jul 06 22:24:39.228621 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:24:39.228480    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 06 22:24:39.228621 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:39.228497    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/5e411fd6-9619-4fb9-a556-f20259e121e0-crio-socket\") pod \"insights-runtime-extractor-jxjpn\" (UID: \"5e411fd6-9619-4fb9-a556-f20259e121e0\") " pod="openshift-insights/insights-runtime-extractor-jxjpn"
Jul 06 22:24:39.228621 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:24:39.228546    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5e411fd6-9619-4fb9-a556-f20259e121e0-insights-runtime-extractor-tls podName:5e411fd6-9619-4fb9-a556-f20259e121e0 nodeName:}" failed. No retries permitted until 2026-07-06 22:24:39.728530001 +0000 UTC m=+45.153576520 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/5e411fd6-9619-4fb9-a556-f20259e121e0-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jxjpn" (UID: "5e411fd6-9619-4fb9-a556-f20259e121e0") : secret "insights-runtime-extractor-tls" not found
Jul 06 22:24:39.228778 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:39.228705    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/5e411fd6-9619-4fb9-a556-f20259e121e0-data-volume\") pod \"insights-runtime-extractor-jxjpn\" (UID: \"5e411fd6-9619-4fb9-a556-f20259e121e0\") " pod="openshift-insights/insights-runtime-extractor-jxjpn"
Jul 06 22:24:39.237377 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:39.237353    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sd288\" (UniqueName: \"kubernetes.io/projected/5e411fd6-9619-4fb9-a556-f20259e121e0-kube-api-access-sd288\") pod \"insights-runtime-extractor-jxjpn\" (UID: \"5e411fd6-9619-4fb9-a556-f20259e121e0\") " pod="openshift-insights/insights-runtime-extractor-jxjpn"
Jul 06 22:24:39.732722 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:39.732682    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5e411fd6-9619-4fb9-a556-f20259e121e0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jxjpn\" (UID: \"5e411fd6-9619-4fb9-a556-f20259e121e0\") " pod="openshift-insights/insights-runtime-extractor-jxjpn"
Jul 06 22:24:39.732722 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:39.732725    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5e411fd6-9619-4fb9-a556-f20259e121e0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jxjpn\" (UID: \"5e411fd6-9619-4fb9-a556-f20259e121e0\") " pod="openshift-insights/insights-runtime-extractor-jxjpn"
Jul 06 22:24:39.732935 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:24:39.732824    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 06 22:24:39.732935 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:24:39.732826    2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 06 22:24:39.732935 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:24:39.732885    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5e411fd6-9619-4fb9-a556-f20259e121e0-insights-runtime-extractor-tls podName:5e411fd6-9619-4fb9-a556-f20259e121e0 nodeName:}" failed. No retries permitted until 2026-07-06 22:24:40.732871282 +0000 UTC m=+46.157917801 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/5e411fd6-9619-4fb9-a556-f20259e121e0-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jxjpn" (UID: "5e411fd6-9619-4fb9-a556-f20259e121e0") : secret "insights-runtime-extractor-tls" not found
Jul 06 22:24:39.732935 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:24:39.732898    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5e411fd6-9619-4fb9-a556-f20259e121e0-kube-rbac-proxy-cm podName:5e411fd6-9619-4fb9-a556-f20259e121e0 nodeName:}" failed. No retries permitted until 2026-07-06 22:24:40.732891708 +0000 UTC m=+46.157938227 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/5e411fd6-9619-4fb9-a556-f20259e121e0-kube-rbac-proxy-cm") pod "insights-runtime-extractor-jxjpn" (UID: "5e411fd6-9619-4fb9-a556-f20259e121e0") : configmap "kube-rbac-proxy" not found
Jul 06 22:24:40.738711 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:40.738675    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5e411fd6-9619-4fb9-a556-f20259e121e0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jxjpn\" (UID: \"5e411fd6-9619-4fb9-a556-f20259e121e0\") " pod="openshift-insights/insights-runtime-extractor-jxjpn"
Jul 06 22:24:40.738711 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:40.738713    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5e411fd6-9619-4fb9-a556-f20259e121e0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jxjpn\" (UID: \"5e411fd6-9619-4fb9-a556-f20259e121e0\") " pod="openshift-insights/insights-runtime-extractor-jxjpn"
Jul 06 22:24:40.739333 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:24:40.738824    2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 06 22:24:40.739333 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:24:40.738852    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 06 22:24:40.739333 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:24:40.738913    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5e411fd6-9619-4fb9-a556-f20259e121e0-kube-rbac-proxy-cm podName:5e411fd6-9619-4fb9-a556-f20259e121e0 nodeName:}" failed. No retries permitted until 2026-07-06 22:24:42.738891436 +0000 UTC m=+48.163937968 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/5e411fd6-9619-4fb9-a556-f20259e121e0-kube-rbac-proxy-cm") pod "insights-runtime-extractor-jxjpn" (UID: "5e411fd6-9619-4fb9-a556-f20259e121e0") : configmap "kube-rbac-proxy" not found
Jul 06 22:24:40.739333 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:24:40.738934    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5e411fd6-9619-4fb9-a556-f20259e121e0-insights-runtime-extractor-tls podName:5e411fd6-9619-4fb9-a556-f20259e121e0 nodeName:}" failed. No retries permitted until 2026-07-06 22:24:42.738923546 +0000 UTC m=+48.163970079 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/5e411fd6-9619-4fb9-a556-f20259e121e0-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jxjpn" (UID: "5e411fd6-9619-4fb9-a556-f20259e121e0") : secret "insights-runtime-extractor-tls" not found
Jul 06 22:24:41.832351 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:41.832323    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-tgnb8_9c27bdd9-eca8-4ef2-a260-56bb21de6b1e/dns-node-resolver/0.log"
Jul 06 22:24:42.630309 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:42.630280    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-9l9l5_6a3050e9-b393-4767-a9c2-dafdf2a17925/node-ca/0.log"
Jul 06 22:24:42.755456 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:42.755422    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5e411fd6-9619-4fb9-a556-f20259e121e0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jxjpn\" (UID: \"5e411fd6-9619-4fb9-a556-f20259e121e0\") " pod="openshift-insights/insights-runtime-extractor-jxjpn"
Jul 06 22:24:42.755456 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:42.755456    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5e411fd6-9619-4fb9-a556-f20259e121e0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jxjpn\" (UID: \"5e411fd6-9619-4fb9-a556-f20259e121e0\") " pod="openshift-insights/insights-runtime-extractor-jxjpn"
Jul 06 22:24:42.755705 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:24:42.755569    2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 06 22:24:42.755705 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:24:42.755577    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 06 22:24:42.755705 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:24:42.755634    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5e411fd6-9619-4fb9-a556-f20259e121e0-insights-runtime-extractor-tls podName:5e411fd6-9619-4fb9-a556-f20259e121e0 nodeName:}" failed. No retries permitted until 2026-07-06 22:24:46.755618781 +0000 UTC m=+52.180665300 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/5e411fd6-9619-4fb9-a556-f20259e121e0-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jxjpn" (UID: "5e411fd6-9619-4fb9-a556-f20259e121e0") : secret "insights-runtime-extractor-tls" not found
Jul 06 22:24:42.755705 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:24:42.755648    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5e411fd6-9619-4fb9-a556-f20259e121e0-kube-rbac-proxy-cm podName:5e411fd6-9619-4fb9-a556-f20259e121e0 nodeName:}" failed. No retries permitted until 2026-07-06 22:24:46.755641321 +0000 UTC m=+52.180687840 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/5e411fd6-9619-4fb9-a556-f20259e121e0-kube-rbac-proxy-cm") pod "insights-runtime-extractor-jxjpn" (UID: "5e411fd6-9619-4fb9-a556-f20259e121e0") : configmap "kube-rbac-proxy" not found
Jul 06 22:24:43.429324 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:43.429299    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-78psz_6e143322-bd8b-42e9-ab2b-7071044779a8/serve-healthcheck-canary/0.log"
Jul 06 22:24:45.781273 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:45.781232    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/62bbe928-f2d0-4642-a2a3-f272341820eb-metrics-tls\") pod \"dns-default-bv5xq\" (UID: \"62bbe928-f2d0-4642-a2a3-f272341820eb\") " pod="openshift-dns/dns-default-bv5xq"
Jul 06 22:24:45.781682 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:24:45.781386    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 06 22:24:45.781682 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:24:45.781459    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/62bbe928-f2d0-4642-a2a3-f272341820eb-metrics-tls podName:62bbe928-f2d0-4642-a2a3-f272341820eb nodeName:}" failed. No retries permitted until 2026-07-06 22:25:01.78144111 +0000 UTC m=+67.206487629 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/62bbe928-f2d0-4642-a2a3-f272341820eb-metrics-tls") pod "dns-default-bv5xq" (UID: "62bbe928-f2d0-4642-a2a3-f272341820eb") : secret "dns-default-metrics-tls" not found
Jul 06 22:24:45.882471 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:45.882433    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/63e27075-6d1f-43a1-82bf-ca8e43da8326-registry-tls\") pod \"image-registry-84f89db6f7-vbq96\" (UID: \"63e27075-6d1f-43a1-82bf-ca8e43da8326\") " pod="openshift-image-registry/image-registry-84f89db6f7-vbq96"
Jul 06 22:24:45.882634 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:24:45.882585    2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 06 22:24:45.882634 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:24:45.882606    2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-84f89db6f7-vbq96: secret "image-registry-tls" not found
Jul 06 22:24:45.882736 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:24:45.882663    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/63e27075-6d1f-43a1-82bf-ca8e43da8326-registry-tls podName:63e27075-6d1f-43a1-82bf-ca8e43da8326 nodeName:}" failed. No retries permitted until 2026-07-06 22:25:01.882648193 +0000 UTC m=+67.307694712 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/63e27075-6d1f-43a1-82bf-ca8e43da8326-registry-tls") pod "image-registry-84f89db6f7-vbq96" (UID: "63e27075-6d1f-43a1-82bf-ca8e43da8326") : secret "image-registry-tls" not found
Jul 06 22:24:46.788907 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:46.788869    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5e411fd6-9619-4fb9-a556-f20259e121e0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jxjpn\" (UID: \"5e411fd6-9619-4fb9-a556-f20259e121e0\") " pod="openshift-insights/insights-runtime-extractor-jxjpn"
Jul 06 22:24:46.788907 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:46.788906    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5e411fd6-9619-4fb9-a556-f20259e121e0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jxjpn\" (UID: \"5e411fd6-9619-4fb9-a556-f20259e121e0\") " pod="openshift-insights/insights-runtime-extractor-jxjpn"
Jul 06 22:24:46.789497 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:24:46.789022    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 06 22:24:46.789497 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:24:46.789079    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5e411fd6-9619-4fb9-a556-f20259e121e0-insights-runtime-extractor-tls podName:5e411fd6-9619-4fb9-a556-f20259e121e0 nodeName:}" failed. No retries permitted until 2026-07-06 22:24:54.789066103 +0000 UTC m=+60.214112622 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/5e411fd6-9619-4fb9-a556-f20259e121e0-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jxjpn" (UID: "5e411fd6-9619-4fb9-a556-f20259e121e0") : secret "insights-runtime-extractor-tls" not found
Jul 06 22:24:46.789626 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:46.789576    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5e411fd6-9619-4fb9-a556-f20259e121e0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jxjpn\" (UID: \"5e411fd6-9619-4fb9-a556-f20259e121e0\") " pod="openshift-insights/insights-runtime-extractor-jxjpn"
Jul 06 22:24:54.849349 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:54.849310    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5e411fd6-9619-4fb9-a556-f20259e121e0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jxjpn\" (UID: \"5e411fd6-9619-4fb9-a556-f20259e121e0\") " pod="openshift-insights/insights-runtime-extractor-jxjpn"
Jul 06 22:24:54.851685 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:54.851653    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5e411fd6-9619-4fb9-a556-f20259e121e0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jxjpn\" (UID: \"5e411fd6-9619-4fb9-a556-f20259e121e0\") " pod="openshift-insights/insights-runtime-extractor-jxjpn"
Jul 06 22:24:54.943508 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:54.943476    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-jxjpn"
Jul 06 22:24:55.105462 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:55.105396    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-jxjpn"]
Jul 06 22:24:55.108353 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:24:55.108322    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e411fd6_9619_4fb9_a556_f20259e121e0.slice/crio-2cf88d1e8c5c901ff33699a6539932f2539bb9ec868bb97b9e509e77e3b6406b WatchSource:0}: Error finding container 2cf88d1e8c5c901ff33699a6539932f2539bb9ec868bb97b9e509e77e3b6406b: Status 404 returned error can't find the container with id 2cf88d1e8c5c901ff33699a6539932f2539bb9ec868bb97b9e509e77e3b6406b
Jul 06 22:24:55.367112 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:55.367020    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jxjpn" event={"ID":"5e411fd6-9619-4fb9-a556-f20259e121e0","Type":"ContainerStarted","Data":"2cf88d1e8c5c901ff33699a6539932f2539bb9ec868bb97b9e509e77e3b6406b"}
Jul 06 22:24:56.301583 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:56.301554    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-t6jpk"
Jul 06 22:24:59.380261 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:59.380228    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jxjpn" event={"ID":"5e411fd6-9619-4fb9-a556-f20259e121e0","Type":"ContainerStarted","Data":"b1a937d86fd960ce59b05bca537ae6990a6209940daef66fd506b717e05ae304"}
Jul 06 22:24:59.790330 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:59.790299    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gvmwj\" (UniqueName: \"kubernetes.io/projected/da82c66a-1f8a-4315-8f45-8b449e2e46b9-kube-api-access-gvmwj\") pod \"network-check-target-6qmpm\" (UID: \"da82c66a-1f8a-4315-8f45-8b449e2e46b9\") " pod="openshift-network-diagnostics/network-check-target-6qmpm"
Jul 06 22:24:59.793090 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:59.793072    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 06 22:24:59.803378 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:59.803359    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 06 22:24:59.813799 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:59.813773    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gvmwj\" (UniqueName: \"kubernetes.io/projected/da82c66a-1f8a-4315-8f45-8b449e2e46b9-kube-api-access-gvmwj\") pod \"network-check-target-6qmpm\" (UID: \"da82c66a-1f8a-4315-8f45-8b449e2e46b9\") " pod="openshift-network-diagnostics/network-check-target-6qmpm"
Jul 06 22:24:59.890934 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:59.890899    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/aad76575-952c-4b85-b64a-2523529c1cf4-original-pull-secret\") pod \"global-pull-secret-syncer-mc48r\" (UID: \"aad76575-952c-4b85-b64a-2523529c1cf4\") " pod="kube-system/global-pull-secret-syncer-mc48r"
Jul 06 22:24:59.891215 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:59.890963    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/69b0b78e-e934-433b-bfdb-5fd6d8ba4a29-metrics-certs\") pod \"network-metrics-daemon-bbjgn\" (UID: \"69b0b78e-e934-433b-bfdb-5fd6d8ba4a29\") " pod="openshift-multus/network-metrics-daemon-bbjgn"
Jul 06 22:24:59.893588 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:59.893557    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 06 22:24:59.893701 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:59.893560    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 06 22:24:59.904792 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:59.904768    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/69b0b78e-e934-433b-bfdb-5fd6d8ba4a29-metrics-certs\") pod \"network-metrics-daemon-bbjgn\" (UID: \"69b0b78e-e934-433b-bfdb-5fd6d8ba4a29\") " pod="openshift-multus/network-metrics-daemon-bbjgn"
Jul 06 22:24:59.905014 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:24:59.904995    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/aad76575-952c-4b85-b64a-2523529c1cf4-original-pull-secret\") pod \"global-pull-secret-syncer-mc48r\" (UID: \"aad76575-952c-4b85-b64a-2523529c1cf4\") " pod="kube-system/global-pull-secret-syncer-mc48r"
Jul 06 22:25:00.088439 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:00.088358    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-zhlqr\""
Jul 06 22:25:00.092573 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:00.092548    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mc48r"
Jul 06 22:25:00.096826 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:00.096801    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bbjgn"
Jul 06 22:25:00.100414 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:00.100397    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-zvxv7\""
Jul 06 22:25:00.108646 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:00.108608    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6qmpm"
Jul 06 22:25:00.254636 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:00.254606    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-bbjgn"]
Jul 06 22:25:00.258183 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:25:00.258130    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69b0b78e_e934_433b_bfdb_5fd6d8ba4a29.slice/crio-d6cabd37c00b797d2ff49829279ebd478f6ea8ffc3a2bcc1cdb85e15f8d3f293 WatchSource:0}: Error finding container d6cabd37c00b797d2ff49829279ebd478f6ea8ffc3a2bcc1cdb85e15f8d3f293: Status 404 returned error can't find the container with id d6cabd37c00b797d2ff49829279ebd478f6ea8ffc3a2bcc1cdb85e15f8d3f293
Jul 06 22:25:00.271624 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:00.271594    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-mc48r"]
Jul 06 22:25:00.274936 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:25:00.274904    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaad76575_952c_4b85_b64a_2523529c1cf4.slice/crio-c388e44565f443cbaeb0ababf2d387457f729641d2bfa357b646298361393ccc WatchSource:0}: Error finding container c388e44565f443cbaeb0ababf2d387457f729641d2bfa357b646298361393ccc: Status 404 returned error can't find the container with id c388e44565f443cbaeb0ababf2d387457f729641d2bfa357b646298361393ccc
Jul 06 22:25:00.291450 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:00.291418    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-6qmpm"]
Jul 06 22:25:00.294554 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:25:00.294525    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda82c66a_1f8a_4315_8f45_8b449e2e46b9.slice/crio-9d0d30c0a61c453b4ab72fa8d72d0d2c46d30d003eea319f940f7304725c9932 WatchSource:0}: Error finding container 9d0d30c0a61c453b4ab72fa8d72d0d2c46d30d003eea319f940f7304725c9932: Status 404 returned error can't find the container with id 9d0d30c0a61c453b4ab72fa8d72d0d2c46d30d003eea319f940f7304725c9932
Jul 06 22:25:00.384874 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:00.384788    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-6qmpm" event={"ID":"da82c66a-1f8a-4315-8f45-8b449e2e46b9","Type":"ContainerStarted","Data":"9d0d30c0a61c453b4ab72fa8d72d0d2c46d30d003eea319f940f7304725c9932"}
Jul 06 22:25:00.385987 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:00.385956    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-mc48r" event={"ID":"aad76575-952c-4b85-b64a-2523529c1cf4","Type":"ContainerStarted","Data":"c388e44565f443cbaeb0ababf2d387457f729641d2bfa357b646298361393ccc"}
Jul 06 22:25:00.387841 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:00.387812    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jxjpn" event={"ID":"5e411fd6-9619-4fb9-a556-f20259e121e0","Type":"ContainerStarted","Data":"2f1ca4508947e86a14582904902dfb401ac9ca15a404ee61c7d63b6728d55c01"}
Jul 06 22:25:00.389071 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:00.389046    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-bbjgn" event={"ID":"69b0b78e-e934-433b-bfdb-5fd6d8ba4a29","Type":"ContainerStarted","Data":"d6cabd37c00b797d2ff49829279ebd478f6ea8ffc3a2bcc1cdb85e15f8d3f293"}
Jul 06 22:25:00.651657 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:00.651568    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b9dcdd858-bg9kf"]
Jul 06 22:25:00.653818 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:00.653791    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b9dcdd858-bg9kf"
Jul 06 22:25:00.659083 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:00.659059    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 06 22:25:00.659220 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:00.659110    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Jul 06 22:25:00.659220 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:00.659059    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 06 22:25:00.659431 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:00.659417    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 06 22:25:00.677020 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:00.676994    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-84f89db6f7-vbq96"]
Jul 06 22:25:00.677232 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:25:00.677212    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context canceled" pod="openshift-image-registry/image-registry-84f89db6f7-vbq96" podUID="63e27075-6d1f-43a1-82bf-ca8e43da8326"
Jul 06 22:25:00.682482 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:00.682454    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b9dcdd858-bg9kf"]
Jul 06 22:25:00.746897 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:00.746827    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-745f45d84c-z5h56"]
Jul 06 22:25:00.749379 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:00.749071    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-745f45d84c-z5h56"
Jul 06 22:25:00.765687 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:00.765666    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-745f45d84c-z5h56"]
Jul 06 22:25:00.797532 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:00.797497    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/9f950050-c621-41d6-ad07-0a0f69b5ac1a-klusterlet-config\") pod \"klusterlet-addon-workmgr-6b9dcdd858-bg9kf\" (UID: \"9f950050-c621-41d6-ad07-0a0f69b5ac1a\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b9dcdd858-bg9kf"
Jul 06 22:25:00.797707 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:00.797588    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9f950050-c621-41d6-ad07-0a0f69b5ac1a-tmp\") pod \"klusterlet-addon-workmgr-6b9dcdd858-bg9kf\" (UID: \"9f950050-c621-41d6-ad07-0a0f69b5ac1a\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b9dcdd858-bg9kf"
Jul 06 22:25:00.797707 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:00.797672    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t7rqr\" (UniqueName: \"kubernetes.io/projected/9f950050-c621-41d6-ad07-0a0f69b5ac1a-kube-api-access-t7rqr\") pod \"klusterlet-addon-workmgr-6b9dcdd858-bg9kf\" (UID: \"9f950050-c621-41d6-ad07-0a0f69b5ac1a\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b9dcdd858-bg9kf"
Jul 06 22:25:00.898965 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:00.898933    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9f950050-c621-41d6-ad07-0a0f69b5ac1a-tmp\") pod \"klusterlet-addon-workmgr-6b9dcdd858-bg9kf\" (UID: \"9f950050-c621-41d6-ad07-0a0f69b5ac1a\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b9dcdd858-bg9kf"
Jul 06 22:25:00.899156 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:00.898995    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/27e84944-f883-4de2-bd21-2577c8e8d01b-image-registry-private-configuration\") pod \"image-registry-745f45d84c-z5h56\" (UID: \"27e84944-f883-4de2-bd21-2577c8e8d01b\") " pod="openshift-image-registry/image-registry-745f45d84c-z5h56"
Jul 06 22:25:00.899156 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:00.899015    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/27e84944-f883-4de2-bd21-2577c8e8d01b-ca-trust-extracted\") pod \"image-registry-745f45d84c-z5h56\" (UID: \"27e84944-f883-4de2-bd21-2577c8e8d01b\") " pod="openshift-image-registry/image-registry-745f45d84c-z5h56"
Jul 06 22:25:00.899156 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:00.899034    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t7rqr\" (UniqueName: \"kubernetes.io/projected/9f950050-c621-41d6-ad07-0a0f69b5ac1a-kube-api-access-t7rqr\") pod \"klusterlet-addon-workmgr-6b9dcdd858-bg9kf\" (UID: \"9f950050-c621-41d6-ad07-0a0f69b5ac1a\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b9dcdd858-bg9kf"
Jul 06 22:25:00.899156 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:00.899051    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/27e84944-f883-4de2-bd21-2577c8e8d01b-trusted-ca\") pod \"image-registry-745f45d84c-z5h56\" (UID: \"27e84944-f883-4de2-bd21-2577c8e8d01b\") " pod="openshift-image-registry/image-registry-745f45d84c-z5h56"
Jul 06 22:25:00.899156 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:00.899075    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/9f950050-c621-41d6-ad07-0a0f69b5ac1a-klusterlet-config\") pod \"klusterlet-addon-workmgr-6b9dcdd858-bg9kf\" (UID: \"9f950050-c621-41d6-ad07-0a0f69b5ac1a\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b9dcdd858-bg9kf"
Jul 06 22:25:00.899156 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:00.899092    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jl67h\" (UniqueName: \"kubernetes.io/projected/27e84944-f883-4de2-bd21-2577c8e8d01b-kube-api-access-jl67h\") pod \"image-registry-745f45d84c-z5h56\" (UID: \"27e84944-f883-4de2-bd21-2577c8e8d01b\") " pod="openshift-image-registry/image-registry-745f45d84c-z5h56"
Jul 06 22:25:00.899156 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:00.899116    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/27e84944-f883-4de2-bd21-2577c8e8d01b-installation-pull-secrets\") pod \"image-registry-745f45d84c-z5h56\" (UID: \"27e84944-f883-4de2-bd21-2577c8e8d01b\") " pod="openshift-image-registry/image-registry-745f45d84c-z5h56"
Jul 06 22:25:00.899156 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:00.899146    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/27e84944-f883-4de2-bd21-2577c8e8d01b-bound-sa-token\") pod \"image-registry-745f45d84c-z5h56\" (UID: \"27e84944-f883-4de2-bd21-2577c8e8d01b\") " pod="openshift-image-registry/image-registry-745f45d84c-z5h56"
Jul 06 22:25:00.899156 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:00.899162    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/27e84944-f883-4de2-bd21-2577c8e8d01b-registry-certificates\") pod \"image-registry-745f45d84c-z5h56\" (UID: \"27e84944-f883-4de2-bd21-2577c8e8d01b\") " pod="openshift-image-registry/image-registry-745f45d84c-z5h56"
Jul 06 22:25:00.899612 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:00.899201    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/27e84944-f883-4de2-bd21-2577c8e8d01b-registry-tls\") pod \"image-registry-745f45d84c-z5h56\" (UID: \"27e84944-f883-4de2-bd21-2577c8e8d01b\") " pod="openshift-image-registry/image-registry-745f45d84c-z5h56"
Jul 06 22:25:00.900459 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:00.899684    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9f950050-c621-41d6-ad07-0a0f69b5ac1a-tmp\") pod \"klusterlet-addon-workmgr-6b9dcdd858-bg9kf\" (UID: \"9f950050-c621-41d6-ad07-0a0f69b5ac1a\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b9dcdd858-bg9kf"
Jul 06 22:25:00.903667 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:00.903590    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/9f950050-c621-41d6-ad07-0a0f69b5ac1a-klusterlet-config\") pod \"klusterlet-addon-workmgr-6b9dcdd858-bg9kf\" (UID: \"9f950050-c621-41d6-ad07-0a0f69b5ac1a\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b9dcdd858-bg9kf"
Jul 06 22:25:00.910936 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:00.910895    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t7rqr\" (UniqueName: \"kubernetes.io/projected/9f950050-c621-41d6-ad07-0a0f69b5ac1a-kube-api-access-t7rqr\") pod \"klusterlet-addon-workmgr-6b9dcdd858-bg9kf\" (UID: \"9f950050-c621-41d6-ad07-0a0f69b5ac1a\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b9dcdd858-bg9kf"
Jul 06 22:25:00.966809 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:00.966768    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b9dcdd858-bg9kf"
Jul 06 22:25:01.007266 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:01.007236    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/27e84944-f883-4de2-bd21-2577c8e8d01b-image-registry-private-configuration\") pod \"image-registry-745f45d84c-z5h56\" (UID: \"27e84944-f883-4de2-bd21-2577c8e8d01b\") " pod="openshift-image-registry/image-registry-745f45d84c-z5h56"
Jul 06 22:25:01.007440 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:01.007275    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/27e84944-f883-4de2-bd21-2577c8e8d01b-ca-trust-extracted\") pod \"image-registry-745f45d84c-z5h56\" (UID: \"27e84944-f883-4de2-bd21-2577c8e8d01b\") " pod="openshift-image-registry/image-registry-745f45d84c-z5h56"
Jul 06 22:25:01.007440 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:01.007301    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/27e84944-f883-4de2-bd21-2577c8e8d01b-trusted-ca\") pod \"image-registry-745f45d84c-z5h56\" (UID: \"27e84944-f883-4de2-bd21-2577c8e8d01b\") " pod="openshift-image-registry/image-registry-745f45d84c-z5h56"
Jul 06 22:25:01.007440 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:01.007327    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jl67h\" (UniqueName: \"kubernetes.io/projected/27e84944-f883-4de2-bd21-2577c8e8d01b-kube-api-access-jl67h\") pod \"image-registry-745f45d84c-z5h56\" (UID: \"27e84944-f883-4de2-bd21-2577c8e8d01b\") " pod="openshift-image-registry/image-registry-745f45d84c-z5h56"
Jul 06 22:25:01.007440 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:01.007362    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/27e84944-f883-4de2-bd21-2577c8e8d01b-installation-pull-secrets\") pod \"image-registry-745f45d84c-z5h56\" (UID: \"27e84944-f883-4de2-bd21-2577c8e8d01b\") " pod="openshift-image-registry/image-registry-745f45d84c-z5h56"
Jul 06 22:25:01.007440 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:01.007384    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/27e84944-f883-4de2-bd21-2577c8e8d01b-bound-sa-token\") pod \"image-registry-745f45d84c-z5h56\" (UID: \"27e84944-f883-4de2-bd21-2577c8e8d01b\") " pod="openshift-image-registry/image-registry-745f45d84c-z5h56"
Jul 06 22:25:01.007440 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:01.007407    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/27e84944-f883-4de2-bd21-2577c8e8d01b-registry-certificates\") pod \"image-registry-745f45d84c-z5h56\" (UID: \"27e84944-f883-4de2-bd21-2577c8e8d01b\") " pod="openshift-image-registry/image-registry-745f45d84c-z5h56"
Jul 06 22:25:01.007440 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:01.007425    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/27e84944-f883-4de2-bd21-2577c8e8d01b-registry-tls\") pod \"image-registry-745f45d84c-z5h56\" (UID: \"27e84944-f883-4de2-bd21-2577c8e8d01b\") " pod="openshift-image-registry/image-registry-745f45d84c-z5h56"
Jul 06 22:25:01.008253 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:01.008091    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/27e84944-f883-4de2-bd21-2577c8e8d01b-ca-trust-extracted\") pod \"image-registry-745f45d84c-z5h56\" (UID: \"27e84944-f883-4de2-bd21-2577c8e8d01b\") " pod="openshift-image-registry/image-registry-745f45d84c-z5h56"
Jul 06 22:25:01.010217 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:01.008871    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/27e84944-f883-4de2-bd21-2577c8e8d01b-trusted-ca\") pod \"image-registry-745f45d84c-z5h56\" (UID: \"27e84944-f883-4de2-bd21-2577c8e8d01b\") " pod="openshift-image-registry/image-registry-745f45d84c-z5h56"
Jul 06 22:25:01.010217 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:01.010122    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/27e84944-f883-4de2-bd21-2577c8e8d01b-registry-certificates\") pod \"image-registry-745f45d84c-z5h56\" (UID: \"27e84944-f883-4de2-bd21-2577c8e8d01b\") " pod="openshift-image-registry/image-registry-745f45d84c-z5h56"
Jul 06 22:25:01.011817 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:01.011668    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/27e84944-f883-4de2-bd21-2577c8e8d01b-registry-tls\") pod \"image-registry-745f45d84c-z5h56\" (UID: \"27e84944-f883-4de2-bd21-2577c8e8d01b\") " pod="openshift-image-registry/image-registry-745f45d84c-z5h56"
Jul 06 22:25:01.012797 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:01.011933    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/27e84944-f883-4de2-bd21-2577c8e8d01b-image-registry-private-configuration\") pod \"image-registry-745f45d84c-z5h56\" (UID: \"27e84944-f883-4de2-bd21-2577c8e8d01b\") " pod="openshift-image-registry/image-registry-745f45d84c-z5h56"
Jul 06 22:25:01.016981 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:01.016954    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/27e84944-f883-4de2-bd21-2577c8e8d01b-installation-pull-secrets\") pod \"image-registry-745f45d84c-z5h56\" (UID: \"27e84944-f883-4de2-bd21-2577c8e8d01b\") " pod="openshift-image-registry/image-registry-745f45d84c-z5h56"
Jul 06 22:25:01.020444 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:01.020403    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/27e84944-f883-4de2-bd21-2577c8e8d01b-bound-sa-token\") pod \"image-registry-745f45d84c-z5h56\" (UID: \"27e84944-f883-4de2-bd21-2577c8e8d01b\") " pod="openshift-image-registry/image-registry-745f45d84c-z5h56"
Jul 06 22:25:01.020552 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:01.020467    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jl67h\" (UniqueName: \"kubernetes.io/projected/27e84944-f883-4de2-bd21-2577c8e8d01b-kube-api-access-jl67h\") pod \"image-registry-745f45d84c-z5h56\" (UID: \"27e84944-f883-4de2-bd21-2577c8e8d01b\") " pod="openshift-image-registry/image-registry-745f45d84c-z5h56"
Jul 06 22:25:01.064468 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:01.064253    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-c4gjv\""
Jul 06 22:25:01.072814 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:01.072561    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-745f45d84c-z5h56"
Jul 06 22:25:01.159604 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:01.159477    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b9dcdd858-bg9kf"]
Jul 06 22:25:01.392244 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:01.392212    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-84f89db6f7-vbq96"
Jul 06 22:25:01.396075 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:25:01.396040    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f950050_c621_41d6_ad07_0a0f69b5ac1a.slice/crio-dc726a13ab8c38ea18d2c4199ddc0b967cb889ebb970404a25dad7f6ae8f0d70 WatchSource:0}: Error finding container dc726a13ab8c38ea18d2c4199ddc0b967cb889ebb970404a25dad7f6ae8f0d70: Status 404 returned error can't find the container with id dc726a13ab8c38ea18d2c4199ddc0b967cb889ebb970404a25dad7f6ae8f0d70
Jul 06 22:25:01.412474 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:01.412190    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-84f89db6f7-vbq96"
Jul 06 22:25:01.511567 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:01.511528    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/63e27075-6d1f-43a1-82bf-ca8e43da8326-bound-sa-token\") pod \"63e27075-6d1f-43a1-82bf-ca8e43da8326\" (UID: \"63e27075-6d1f-43a1-82bf-ca8e43da8326\") "
Jul 06 22:25:01.511567 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:01.511568    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/63e27075-6d1f-43a1-82bf-ca8e43da8326-image-registry-private-configuration\") pod \"63e27075-6d1f-43a1-82bf-ca8e43da8326\" (UID: \"63e27075-6d1f-43a1-82bf-ca8e43da8326\") "
Jul 06 22:25:01.511806 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:01.511602    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/63e27075-6d1f-43a1-82bf-ca8e43da8326-ca-trust-extracted\") pod \"63e27075-6d1f-43a1-82bf-ca8e43da8326\" (UID: \"63e27075-6d1f-43a1-82bf-ca8e43da8326\") "
Jul 06 22:25:01.511806 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:01.511622    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k6sd7\" (UniqueName: \"kubernetes.io/projected/63e27075-6d1f-43a1-82bf-ca8e43da8326-kube-api-access-k6sd7\") pod \"63e27075-6d1f-43a1-82bf-ca8e43da8326\" (UID: \"63e27075-6d1f-43a1-82bf-ca8e43da8326\") "
Jul 06 22:25:01.511806 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:01.511677    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/63e27075-6d1f-43a1-82bf-ca8e43da8326-registry-certificates\") pod \"63e27075-6d1f-43a1-82bf-ca8e43da8326\" (UID: \"63e27075-6d1f-43a1-82bf-ca8e43da8326\") "
Jul 06 22:25:01.511806 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:01.511704    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/63e27075-6d1f-43a1-82bf-ca8e43da8326-trusted-ca\") pod \"63e27075-6d1f-43a1-82bf-ca8e43da8326\" (UID: \"63e27075-6d1f-43a1-82bf-ca8e43da8326\") "
Jul 06 22:25:01.511806 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:01.511761    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/63e27075-6d1f-43a1-82bf-ca8e43da8326-installation-pull-secrets\") pod \"63e27075-6d1f-43a1-82bf-ca8e43da8326\" (UID: \"63e27075-6d1f-43a1-82bf-ca8e43da8326\") "
Jul 06 22:25:01.513083 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:01.512453    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/63e27075-6d1f-43a1-82bf-ca8e43da8326-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "63e27075-6d1f-43a1-82bf-ca8e43da8326" (UID: "63e27075-6d1f-43a1-82bf-ca8e43da8326"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 22:25:01.513083 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:01.512985    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/63e27075-6d1f-43a1-82bf-ca8e43da8326-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "63e27075-6d1f-43a1-82bf-ca8e43da8326" (UID: "63e27075-6d1f-43a1-82bf-ca8e43da8326"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 22:25:01.513083 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:01.513036    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/63e27075-6d1f-43a1-82bf-ca8e43da8326-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "63e27075-6d1f-43a1-82bf-ca8e43da8326" (UID: "63e27075-6d1f-43a1-82bf-ca8e43da8326"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 22:25:01.515191 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:01.515132    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/63e27075-6d1f-43a1-82bf-ca8e43da8326-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "63e27075-6d1f-43a1-82bf-ca8e43da8326" (UID: "63e27075-6d1f-43a1-82bf-ca8e43da8326"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 22:25:01.517036 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:01.516999    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/63e27075-6d1f-43a1-82bf-ca8e43da8326-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "63e27075-6d1f-43a1-82bf-ca8e43da8326" (UID: "63e27075-6d1f-43a1-82bf-ca8e43da8326"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 22:25:01.517616 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:01.517575    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/63e27075-6d1f-43a1-82bf-ca8e43da8326-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "63e27075-6d1f-43a1-82bf-ca8e43da8326" (UID: "63e27075-6d1f-43a1-82bf-ca8e43da8326"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 22:25:01.520915 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:01.520889    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/63e27075-6d1f-43a1-82bf-ca8e43da8326-kube-api-access-k6sd7" (OuterVolumeSpecName: "kube-api-access-k6sd7") pod "63e27075-6d1f-43a1-82bf-ca8e43da8326" (UID: "63e27075-6d1f-43a1-82bf-ca8e43da8326"). InnerVolumeSpecName "kube-api-access-k6sd7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 22:25:01.612590 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:01.612553    2563 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/63e27075-6d1f-43a1-82bf-ca8e43da8326-bound-sa-token\") on node \"ip-10-0-133-105.ec2.internal\" DevicePath \"\""
Jul 06 22:25:01.612590 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:01.612594    2563 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/63e27075-6d1f-43a1-82bf-ca8e43da8326-image-registry-private-configuration\") on node \"ip-10-0-133-105.ec2.internal\" DevicePath \"\""
Jul 06 22:25:01.612774 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:01.612611    2563 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/63e27075-6d1f-43a1-82bf-ca8e43da8326-ca-trust-extracted\") on node \"ip-10-0-133-105.ec2.internal\" DevicePath \"\""
Jul 06 22:25:01.612774 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:01.612627    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k6sd7\" (UniqueName: \"kubernetes.io/projected/63e27075-6d1f-43a1-82bf-ca8e43da8326-kube-api-access-k6sd7\") on node \"ip-10-0-133-105.ec2.internal\" DevicePath \"\""
Jul 06 22:25:01.612774 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:01.612641    2563 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/63e27075-6d1f-43a1-82bf-ca8e43da8326-registry-certificates\") on node \"ip-10-0-133-105.ec2.internal\" DevicePath \"\""
Jul 06 22:25:01.612774 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:01.612654    2563 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/63e27075-6d1f-43a1-82bf-ca8e43da8326-trusted-ca\") on node \"ip-10-0-133-105.ec2.internal\" DevicePath \"\""
Jul 06 22:25:01.612774 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:01.612667    2563 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/63e27075-6d1f-43a1-82bf-ca8e43da8326-installation-pull-secrets\") on node \"ip-10-0-133-105.ec2.internal\" DevicePath \"\""
Jul 06 22:25:01.814321 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:01.814210    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/62bbe928-f2d0-4642-a2a3-f272341820eb-metrics-tls\") pod \"dns-default-bv5xq\" (UID: \"62bbe928-f2d0-4642-a2a3-f272341820eb\") " pod="openshift-dns/dns-default-bv5xq"
Jul 06 22:25:01.820369 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:01.820309    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/62bbe928-f2d0-4642-a2a3-f272341820eb-metrics-tls\") pod \"dns-default-bv5xq\" (UID: \"62bbe928-f2d0-4642-a2a3-f272341820eb\") " pod="openshift-dns/dns-default-bv5xq"
Jul 06 22:25:01.915651 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:01.915611    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/63e27075-6d1f-43a1-82bf-ca8e43da8326-registry-tls\") pod \"image-registry-84f89db6f7-vbq96\" (UID: \"63e27075-6d1f-43a1-82bf-ca8e43da8326\") " pod="openshift-image-registry/image-registry-84f89db6f7-vbq96"
Jul 06 22:25:01.918697 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:01.918644    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/63e27075-6d1f-43a1-82bf-ca8e43da8326-registry-tls\") pod \"image-registry-84f89db6f7-vbq96\" (UID: \"63e27075-6d1f-43a1-82bf-ca8e43da8326\") " pod="openshift-image-registry/image-registry-84f89db6f7-vbq96"
Jul 06 22:25:02.015980 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:02.015950    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/63e27075-6d1f-43a1-82bf-ca8e43da8326-registry-tls\") pod \"63e27075-6d1f-43a1-82bf-ca8e43da8326\" (UID: \"63e27075-6d1f-43a1-82bf-ca8e43da8326\") "
Jul 06 22:25:02.019036 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:02.018998    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/63e27075-6d1f-43a1-82bf-ca8e43da8326-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "63e27075-6d1f-43a1-82bf-ca8e43da8326" (UID: "63e27075-6d1f-43a1-82bf-ca8e43da8326"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 22:25:02.083329 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:02.083080    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-2cqxg\""
Jul 06 22:25:02.091358 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:02.091149    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-bv5xq"
Jul 06 22:25:02.116378 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:02.116349    2563 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/63e27075-6d1f-43a1-82bf-ca8e43da8326-registry-tls\") on node \"ip-10-0-133-105.ec2.internal\" DevicePath \"\""
Jul 06 22:25:02.398693 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:02.398624    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-84f89db6f7-vbq96"
Jul 06 22:25:02.399381 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:02.399341    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b9dcdd858-bg9kf" event={"ID":"9f950050-c621-41d6-ad07-0a0f69b5ac1a","Type":"ContainerStarted","Data":"dc726a13ab8c38ea18d2c4199ddc0b967cb889ebb970404a25dad7f6ae8f0d70"}
Jul 06 22:25:02.437324 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:02.437292    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-84f89db6f7-vbq96"]
Jul 06 22:25:02.440899 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:02.440877    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-84f89db6f7-vbq96"]
Jul 06 22:25:03.078901 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:03.078862    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="63e27075-6d1f-43a1-82bf-ca8e43da8326" path="/var/lib/kubelet/pods/63e27075-6d1f-43a1-82bf-ca8e43da8326/volumes"
Jul 06 22:25:06.513799 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:06.513685    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-745f45d84c-z5h56"]
Jul 06 22:25:06.517067 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:06.516801    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-bv5xq"]
Jul 06 22:25:07.377600 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:07.377569    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-btxqr"]
Jul 06 22:25:07.379765 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:07.379731    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-btxqr"
Jul 06 22:25:07.382156 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:07.382134    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Jul 06 22:25:07.382285 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:07.382239    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-dg2gb\""
Jul 06 22:25:07.391255 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:07.391227    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-btxqr"]
Jul 06 22:25:07.431403 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:07.431365    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-6qmpm" event={"ID":"da82c66a-1f8a-4315-8f45-8b449e2e46b9","Type":"ContainerStarted","Data":"424e5d0c28cf4ab68bcd26801cbe898dc92ef342c9380a0a67443ba22a38aa1f"}
Jul 06 22:25:07.431570 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:07.431548    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-6qmpm"
Jul 06 22:25:07.433327 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:07.433290    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-mc48r" event={"ID":"aad76575-952c-4b85-b64a-2523529c1cf4","Type":"ContainerStarted","Data":"3b1b09580f3d705aa788b886b4f9b4fbbc3d3c1c9a2c88e6e84a63e900f66274"}
Jul 06 22:25:07.435922 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:07.435896    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jxjpn" event={"ID":"5e411fd6-9619-4fb9-a556-f20259e121e0","Type":"ContainerStarted","Data":"3c4ce64495d2dfd14419f83594dea73077494a34cf69d26334c88fed36fa10b5"}
Jul 06 22:25:07.437635 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:07.437607    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-bbjgn" event={"ID":"69b0b78e-e934-433b-bfdb-5fd6d8ba4a29","Type":"ContainerStarted","Data":"5864b36787e5f42af2a12c8a2397a2632de3059b8ebe27d189a9b4db492d4488"}
Jul 06 22:25:07.448872 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:07.448834    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-6qmpm" podStartSLOduration=66.42822793 podStartE2EDuration="1m12.448819491s" totalImagesPullingTime="6.020591561s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:23:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:25:07.448119379 +0000 UTC m=+72.873165920" watchObservedRunningTime="2026-07-06 22:25:07.448819491 +0000 UTC m=+72.873866031"
Jul 06 22:25:07.453145 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:07.453119    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/080c7027-a067-4d22-9bff-c818bca84503-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-btxqr\" (UID: \"080c7027-a067-4d22-9bff-c818bca84503\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-btxqr"
Jul 06 22:25:07.465290 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:07.465247    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-jxjpn" podStartSLOduration=17.354794685 podStartE2EDuration="28.465232044s" totalImagesPullingTime="11.110437359s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 22:24:39 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:25:07.464629571 +0000 UTC m=+72.889676126" watchObservedRunningTime="2026-07-06 22:25:07.465232044 +0000 UTC m=+72.890278562"
Jul 06 22:25:07.481587 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:07.481529    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-mc48r" podStartSLOduration=66.420122157 podStartE2EDuration="1m12.481513844s" totalImagesPullingTime="6.061391687s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 22:23:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:25:07.480619709 +0000 UTC m=+72.905666253" watchObservedRunningTime="2026-07-06 22:25:07.481513844 +0000 UTC m=+72.906560384"
Jul 06 22:25:07.533057 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:07.533020    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-685bfb5f64-lgzh4"]
Jul 06 22:25:07.535274 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:07.535254    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-685bfb5f64-lgzh4"
Jul 06 22:25:07.537834 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:07.537710    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-xlt6p\""
Jul 06 22:25:07.537834 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:07.537710    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 06 22:25:07.538425 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:07.538405    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 06 22:25:07.548971 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:07.548950    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-685bfb5f64-lgzh4"]
Jul 06 22:25:07.554220 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:07.554196    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kllmx\" (UniqueName: \"kubernetes.io/projected/11551149-01c0-4656-be37-d24965aac4a4-kube-api-access-kllmx\") pod \"downloads-685bfb5f64-lgzh4\" (UID: \"11551149-01c0-4656-be37-d24965aac4a4\") " pod="openshift-console/downloads-685bfb5f64-lgzh4"
Jul 06 22:25:07.554326 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:07.554244    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/080c7027-a067-4d22-9bff-c818bca84503-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-btxqr\" (UID: \"080c7027-a067-4d22-9bff-c818bca84503\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-btxqr"
Jul 06 22:25:07.556867 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:07.556840    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/080c7027-a067-4d22-9bff-c818bca84503-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-btxqr\" (UID: \"080c7027-a067-4d22-9bff-c818bca84503\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-btxqr"
Jul 06 22:25:07.645128 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:25:07.645096    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27e84944_f883_4de2_bd21_2577c8e8d01b.slice/crio-8867653d798d852e35ad13693356f28ba9ef4110c42e6b5efc0254817884f178 WatchSource:0}: Error finding container 8867653d798d852e35ad13693356f28ba9ef4110c42e6b5efc0254817884f178: Status 404 returned error can't find the container with id 8867653d798d852e35ad13693356f28ba9ef4110c42e6b5efc0254817884f178
Jul 06 22:25:07.645808 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:25:07.645740    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62bbe928_f2d0_4642_a2a3_f272341820eb.slice/crio-799627d2f5584472cea336e6dd9578c4f63de4f64bd1015b1a6c60cca66968ca WatchSource:0}: Error finding container 799627d2f5584472cea336e6dd9578c4f63de4f64bd1015b1a6c60cca66968ca: Status 404 returned error can't find the container with id 799627d2f5584472cea336e6dd9578c4f63de4f64bd1015b1a6c60cca66968ca
Jul 06 22:25:07.654757 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:07.654729    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kllmx\" (UniqueName: \"kubernetes.io/projected/11551149-01c0-4656-be37-d24965aac4a4-kube-api-access-kllmx\") pod \"downloads-685bfb5f64-lgzh4\" (UID: \"11551149-01c0-4656-be37-d24965aac4a4\") " pod="openshift-console/downloads-685bfb5f64-lgzh4"
Jul 06 22:25:07.665894 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:07.665871    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kllmx\" (UniqueName: \"kubernetes.io/projected/11551149-01c0-4656-be37-d24965aac4a4-kube-api-access-kllmx\") pod \"downloads-685bfb5f64-lgzh4\" (UID: \"11551149-01c0-4656-be37-d24965aac4a4\") " pod="openshift-console/downloads-685bfb5f64-lgzh4"
Jul 06 22:25:07.691643 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:07.691617    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-btxqr"
Jul 06 22:25:07.831794 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:07.831762    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-btxqr"]
Jul 06 22:25:07.835297 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:25:07.835266    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod080c7027_a067_4d22_9bff_c818bca84503.slice/crio-12e0659f13a820e49b74272292c59fa4e39e0220a8868f48718bc5530a65af1b WatchSource:0}: Error finding container 12e0659f13a820e49b74272292c59fa4e39e0220a8868f48718bc5530a65af1b: Status 404 returned error can't find the container with id 12e0659f13a820e49b74272292c59fa4e39e0220a8868f48718bc5530a65af1b
Jul 06 22:25:07.847113 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:07.847089    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-685bfb5f64-lgzh4"
Jul 06 22:25:07.976471 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:07.976440    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-685bfb5f64-lgzh4"]
Jul 06 22:25:07.979652 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:25:07.979611    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11551149_01c0_4656_be37_d24965aac4a4.slice/crio-53345d4d0550c0fe8fe8b770386d7ef8db009641927d0f34a6172213d3c2cd5a WatchSource:0}: Error finding container 53345d4d0550c0fe8fe8b770386d7ef8db009641927d0f34a6172213d3c2cd5a: Status 404 returned error can't find the container with id 53345d4d0550c0fe8fe8b770386d7ef8db009641927d0f34a6172213d3c2cd5a
Jul 06 22:25:08.445153 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:08.445095    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-bbjgn" event={"ID":"69b0b78e-e934-433b-bfdb-5fd6d8ba4a29","Type":"ContainerStarted","Data":"097327f9f9bb647335f0ca597d99934c59a90f5699bbf49dfcfa295854eb42cb"}
Jul 06 22:25:08.446856 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:08.446825    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b9dcdd858-bg9kf" event={"ID":"9f950050-c621-41d6-ad07-0a0f69b5ac1a","Type":"ContainerStarted","Data":"e04a5113fb08f31a7286920f45296404bd630cde6eb721f99c4cbf036b728ffe"}
Jul 06 22:25:08.447008 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:08.446989    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b9dcdd858-bg9kf"
Jul 06 22:25:08.448382 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:08.448355    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-btxqr" event={"ID":"080c7027-a067-4d22-9bff-c818bca84503","Type":"ContainerStarted","Data":"12e0659f13a820e49b74272292c59fa4e39e0220a8868f48718bc5530a65af1b"}
Jul 06 22:25:08.448846 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:08.448803    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b9dcdd858-bg9kf"
Jul 06 22:25:08.449576 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:08.449546    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-bv5xq" event={"ID":"62bbe928-f2d0-4642-a2a3-f272341820eb","Type":"ContainerStarted","Data":"799627d2f5584472cea336e6dd9578c4f63de4f64bd1015b1a6c60cca66968ca"}
Jul 06 22:25:08.450755 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:08.450734    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-685bfb5f64-lgzh4" event={"ID":"11551149-01c0-4656-be37-d24965aac4a4","Type":"ContainerStarted","Data":"53345d4d0550c0fe8fe8b770386d7ef8db009641927d0f34a6172213d3c2cd5a"}
Jul 06 22:25:08.452405 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:08.452382    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-745f45d84c-z5h56" event={"ID":"27e84944-f883-4de2-bd21-2577c8e8d01b","Type":"ContainerStarted","Data":"5e2c6c5c7b244b979a218ad7da37510f5d5a6be8314351e616b277fb314812be"}
Jul 06 22:25:08.452790 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:08.452771    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-745f45d84c-z5h56"
Jul 06 22:25:08.452860 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:08.452809    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-745f45d84c-z5h56" event={"ID":"27e84944-f883-4de2-bd21-2577c8e8d01b","Type":"ContainerStarted","Data":"8867653d798d852e35ad13693356f28ba9ef4110c42e6b5efc0254817884f178"}
Jul 06 22:25:08.464582 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:08.464534    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-bbjgn" podStartSLOduration=67.429962337 podStartE2EDuration="1m13.464521034s" totalImagesPullingTime="6.034558697s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:23:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:25:08.463408499 +0000 UTC m=+73.888455041" watchObservedRunningTime="2026-07-06 22:25:08.464521034 +0000 UTC m=+73.889567612"
Jul 06 22:25:08.481739 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:08.481700    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b9dcdd858-bg9kf" podStartSLOduration=2.180885865 podStartE2EDuration="8.481682602s" totalImagesPullingTime="6.300796737s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:25:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:25:08.480647113 +0000 UTC m=+73.905693656" watchObservedRunningTime="2026-07-06 22:25:08.481682602 +0000 UTC m=+73.906729122"
Jul 06 22:25:08.501924 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:08.501869    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-745f45d84c-z5h56" podStartSLOduration=8.501854019 podStartE2EDuration="8.501854019s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:25:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:25:08.500439306 +0000 UTC m=+73.925485847" watchObservedRunningTime="2026-07-06 22:25:08.501854019 +0000 UTC m=+73.926900560"
Jul 06 22:25:10.468381 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:10.468338    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-bv5xq" event={"ID":"62bbe928-f2d0-4642-a2a3-f272341820eb","Type":"ContainerStarted","Data":"f7ecba2075f6e7ca0f41fede6bd22f9468e5277c0748b6d63269828dae11f0d4"}
Jul 06 22:25:10.468850 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:10.468437    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-bv5xq"
Jul 06 22:25:10.468850 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:10.468456    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-bv5xq" event={"ID":"62bbe928-f2d0-4642-a2a3-f272341820eb","Type":"ContainerStarted","Data":"b7f796876601893ce60ebbdcec0018bdb7beb381b19268f92ca20a7681fba88d"}
Jul 06 22:25:10.470157 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:10.470133    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-btxqr" event={"ID":"080c7027-a067-4d22-9bff-c818bca84503","Type":"ContainerStarted","Data":"b030892d2764fd604987804ccb1914fb7c47147f51a32bc2dba19ab8c8ebcf1e"}
Jul 06 22:25:10.470346 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:10.470331    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-btxqr"
Jul 06 22:25:10.475754 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:10.475733    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-btxqr"
Jul 06 22:25:10.485799 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:10.485753    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-bv5xq" podStartSLOduration=39.665371734 podStartE2EDuration="41.485738051s" totalImagesPullingTime="1.820366317s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:24:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:25:10.484952213 +0000 UTC m=+75.909998753" watchObservedRunningTime="2026-07-06 22:25:10.485738051 +0000 UTC m=+75.910784593"
Jul 06 22:25:10.499199 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:10.499141    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-btxqr" podStartSLOduration=1.869492978 podStartE2EDuration="3.499128935s" totalImagesPullingTime="1.629635957s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:25:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:25:10.49813177 +0000 UTC m=+75.923178312" watchObservedRunningTime="2026-07-06 22:25:10.499128935 +0000 UTC m=+75.924175477"
Jul 06 22:25:11.447308 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:11.447266    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-hkrrp"]
Jul 06 22:25:11.449563 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:11.449542    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-hkrrp"
Jul 06 22:25:11.452114 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:11.452085    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 06 22:25:11.452354 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:11.452339    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Jul 06 22:25:11.453384 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:11.453188    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 06 22:25:11.453384 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:11.453240    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Jul 06 22:25:11.453384 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:11.453265    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-hfc9k\""
Jul 06 22:25:11.453384 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:11.453246    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 06 22:25:11.460952 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:11.460932    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-hkrrp"]
Jul 06 22:25:11.488671 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:11.488642    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/b95fccd2-4822-4419-b9ba-51764b5edfc6-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-hkrrp\" (UID: \"b95fccd2-4822-4419-b9ba-51764b5edfc6\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-hkrrp"
Jul 06 22:25:11.489013 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:11.488679    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b95fccd2-4822-4419-b9ba-51764b5edfc6-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-hkrrp\" (UID: \"b95fccd2-4822-4419-b9ba-51764b5edfc6\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-hkrrp"
Jul 06 22:25:11.489013 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:11.488725    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q4wz6\" (UniqueName: \"kubernetes.io/projected/b95fccd2-4822-4419-b9ba-51764b5edfc6-kube-api-access-q4wz6\") pod \"prometheus-operator-567f7b8b7c-hkrrp\" (UID: \"b95fccd2-4822-4419-b9ba-51764b5edfc6\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-hkrrp"
Jul 06 22:25:11.489013 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:11.488795    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b95fccd2-4822-4419-b9ba-51764b5edfc6-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-hkrrp\" (UID: \"b95fccd2-4822-4419-b9ba-51764b5edfc6\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-hkrrp"
Jul 06 22:25:11.589523 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:11.589480    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q4wz6\" (UniqueName: \"kubernetes.io/projected/b95fccd2-4822-4419-b9ba-51764b5edfc6-kube-api-access-q4wz6\") pod \"prometheus-operator-567f7b8b7c-hkrrp\" (UID: \"b95fccd2-4822-4419-b9ba-51764b5edfc6\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-hkrrp"
Jul 06 22:25:11.589692 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:11.589539    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b95fccd2-4822-4419-b9ba-51764b5edfc6-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-hkrrp\" (UID: \"b95fccd2-4822-4419-b9ba-51764b5edfc6\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-hkrrp"
Jul 06 22:25:11.589692 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:11.589600    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/b95fccd2-4822-4419-b9ba-51764b5edfc6-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-hkrrp\" (UID: \"b95fccd2-4822-4419-b9ba-51764b5edfc6\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-hkrrp"
Jul 06 22:25:11.589692 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:11.589628    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b95fccd2-4822-4419-b9ba-51764b5edfc6-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-hkrrp\" (UID: \"b95fccd2-4822-4419-b9ba-51764b5edfc6\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-hkrrp"
Jul 06 22:25:11.590561 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:11.590533    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b95fccd2-4822-4419-b9ba-51764b5edfc6-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-hkrrp\" (UID: \"b95fccd2-4822-4419-b9ba-51764b5edfc6\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-hkrrp"
Jul 06 22:25:11.592881 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:11.592852    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b95fccd2-4822-4419-b9ba-51764b5edfc6-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-hkrrp\" (UID: \"b95fccd2-4822-4419-b9ba-51764b5edfc6\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-hkrrp"
Jul 06 22:25:11.593120 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:11.593095    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/b95fccd2-4822-4419-b9ba-51764b5edfc6-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-hkrrp\" (UID: \"b95fccd2-4822-4419-b9ba-51764b5edfc6\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-hkrrp"
Jul 06 22:25:11.599803 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:11.599780    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q4wz6\" (UniqueName: \"kubernetes.io/projected/b95fccd2-4822-4419-b9ba-51764b5edfc6-kube-api-access-q4wz6\") pod \"prometheus-operator-567f7b8b7c-hkrrp\" (UID: \"b95fccd2-4822-4419-b9ba-51764b5edfc6\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-hkrrp"
Jul 06 22:25:11.761433 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:11.761357    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-hkrrp"
Jul 06 22:25:11.903388 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:11.903359    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-hkrrp"]
Jul 06 22:25:11.906718 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:25:11.906688    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb95fccd2_4822_4419_b9ba_51764b5edfc6.slice/crio-0d31f3fafa2b4329e8453954437296b7049b36c62526e8ffac230ed0014cf862 WatchSource:0}: Error finding container 0d31f3fafa2b4329e8453954437296b7049b36c62526e8ffac230ed0014cf862: Status 404 returned error can't find the container with id 0d31f3fafa2b4329e8453954437296b7049b36c62526e8ffac230ed0014cf862
Jul 06 22:25:12.479702 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:12.479664    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-hkrrp" event={"ID":"b95fccd2-4822-4419-b9ba-51764b5edfc6","Type":"ContainerStarted","Data":"0d31f3fafa2b4329e8453954437296b7049b36c62526e8ffac230ed0014cf862"}
Jul 06 22:25:13.485928 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:13.485887    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-hkrrp" event={"ID":"b95fccd2-4822-4419-b9ba-51764b5edfc6","Type":"ContainerStarted","Data":"f7be67d86e8501735e35923320361fcfa001b7d481c6b83da0397f031e70eeb2"}
Jul 06 22:25:13.486369 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:13.485975    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-hkrrp" event={"ID":"b95fccd2-4822-4419-b9ba-51764b5edfc6","Type":"ContainerStarted","Data":"57b211d8f80085103b3b3dbea4a81f0bd488c894c3f0c66b75fc707fdd8e2edd"}
Jul 06 22:25:15.884783 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:15.884722    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-hkrrp" podStartSLOduration=3.496691995 podStartE2EDuration="4.884696509s" totalImagesPullingTime="1.388004514s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:25:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:25:13.501825288 +0000 UTC m=+78.926871830" watchObservedRunningTime="2026-07-06 22:25:15.884696509 +0000 UTC m=+81.309743051"
Jul 06 22:25:15.885282 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:15.885263    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-gzfm7"]
Jul 06 22:25:15.888267 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:15.888245    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-gzfm7"
Jul 06 22:25:15.891571 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:15.891550    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 06 22:25:15.892438 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:15.892413    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 06 22:25:15.892561 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:15.892457    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-wsr9r\""
Jul 06 22:25:15.892561 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:15.892484    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 06 22:25:15.910367 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:15.910334    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-zldgb"]
Jul 06 22:25:15.912990 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:15.912972    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-zldgb"
Jul 06 22:25:15.915571 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:15.915552    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Jul 06 22:25:15.916015 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:15.915904    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-s8drl\""
Jul 06 22:25:15.916478 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:15.916458    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Jul 06 22:25:15.916613 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:15.916594    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Jul 06 22:25:15.921768 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:15.921749    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/27c8bc73-d90a-4cee-b0f4-28c7b6d06012-root\") pod \"node-exporter-gzfm7\" (UID: \"27c8bc73-d90a-4cee-b0f4-28c7b6d06012\") " pod="openshift-monitoring/node-exporter-gzfm7"
Jul 06 22:25:15.921861 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:15.921803    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/27c8bc73-d90a-4cee-b0f4-28c7b6d06012-sys\") pod \"node-exporter-gzfm7\" (UID: \"27c8bc73-d90a-4cee-b0f4-28c7b6d06012\") " pod="openshift-monitoring/node-exporter-gzfm7"
Jul 06 22:25:15.921861 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:15.921851    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/27c8bc73-d90a-4cee-b0f4-28c7b6d06012-metrics-client-ca\") pod \"node-exporter-gzfm7\" (UID: \"27c8bc73-d90a-4cee-b0f4-28c7b6d06012\") " pod="openshift-monitoring/node-exporter-gzfm7"
Jul 06 22:25:15.921965 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:15.921880    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/27c8bc73-d90a-4cee-b0f4-28c7b6d06012-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-gzfm7\" (UID: \"27c8bc73-d90a-4cee-b0f4-28c7b6d06012\") " pod="openshift-monitoring/node-exporter-gzfm7"
Jul 06 22:25:15.921965 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:15.921905    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kdqzm\" (UniqueName: \"kubernetes.io/projected/27c8bc73-d90a-4cee-b0f4-28c7b6d06012-kube-api-access-kdqzm\") pod \"node-exporter-gzfm7\" (UID: \"27c8bc73-d90a-4cee-b0f4-28c7b6d06012\") " pod="openshift-monitoring/node-exporter-gzfm7"
Jul 06 22:25:15.921965 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:15.921935    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/27c8bc73-d90a-4cee-b0f4-28c7b6d06012-node-exporter-wtmp\") pod \"node-exporter-gzfm7\" (UID: \"27c8bc73-d90a-4cee-b0f4-28c7b6d06012\") " pod="openshift-monitoring/node-exporter-gzfm7"
Jul 06 22:25:15.922123 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:15.922001    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/27c8bc73-d90a-4cee-b0f4-28c7b6d06012-node-exporter-textfile\") pod \"node-exporter-gzfm7\" (UID: \"27c8bc73-d90a-4cee-b0f4-28c7b6d06012\") " pod="openshift-monitoring/node-exporter-gzfm7"
Jul 06 22:25:15.922123 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:15.922041    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/27c8bc73-d90a-4cee-b0f4-28c7b6d06012-node-exporter-tls\") pod \"node-exporter-gzfm7\" (UID: \"27c8bc73-d90a-4cee-b0f4-28c7b6d06012\") " pod="openshift-monitoring/node-exporter-gzfm7"
Jul 06 22:25:15.922123 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:15.922082    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/27c8bc73-d90a-4cee-b0f4-28c7b6d06012-node-exporter-accelerators-collector-config\") pod \"node-exporter-gzfm7\" (UID: \"27c8bc73-d90a-4cee-b0f4-28c7b6d06012\") " pod="openshift-monitoring/node-exporter-gzfm7"
Jul 06 22:25:15.939584 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:15.939549    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-zldgb"]
Jul 06 22:25:16.022830 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:16.022688    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/27c8bc73-d90a-4cee-b0f4-28c7b6d06012-metrics-client-ca\") pod \"node-exporter-gzfm7\" (UID: \"27c8bc73-d90a-4cee-b0f4-28c7b6d06012\") " pod="openshift-monitoring/node-exporter-gzfm7"
Jul 06 22:25:16.022830 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:16.022734    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/f098e977-5c44-4797-b931-19ab8d0165a7-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-zldgb\" (UID: \"f098e977-5c44-4797-b931-19ab8d0165a7\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-zldgb"
Jul 06 22:25:16.022830 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:16.022767    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f098e977-5c44-4797-b931-19ab8d0165a7-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-zldgb\" (UID: \"f098e977-5c44-4797-b931-19ab8d0165a7\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-zldgb"
Jul 06 22:25:16.023125 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:16.022831    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/27c8bc73-d90a-4cee-b0f4-28c7b6d06012-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-gzfm7\" (UID: \"27c8bc73-d90a-4cee-b0f4-28c7b6d06012\") " pod="openshift-monitoring/node-exporter-gzfm7"
Jul 06 22:25:16.023125 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:16.022872    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f098e977-5c44-4797-b931-19ab8d0165a7-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-zldgb\" (UID: \"f098e977-5c44-4797-b931-19ab8d0165a7\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-zldgb"
Jul 06 22:25:16.023125 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:16.022903    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kdqzm\" (UniqueName: \"kubernetes.io/projected/27c8bc73-d90a-4cee-b0f4-28c7b6d06012-kube-api-access-kdqzm\") pod \"node-exporter-gzfm7\" (UID: \"27c8bc73-d90a-4cee-b0f4-28c7b6d06012\") " pod="openshift-monitoring/node-exporter-gzfm7"
Jul 06 22:25:16.023125 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:16.022942    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/f098e977-5c44-4797-b931-19ab8d0165a7-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-zldgb\" (UID: \"f098e977-5c44-4797-b931-19ab8d0165a7\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-zldgb"
Jul 06 22:25:16.023125 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:16.022971    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fgjdl\" (UniqueName: \"kubernetes.io/projected/f098e977-5c44-4797-b931-19ab8d0165a7-kube-api-access-fgjdl\") pod \"kube-state-metrics-7c9dbb8b7d-zldgb\" (UID: \"f098e977-5c44-4797-b931-19ab8d0165a7\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-zldgb"
Jul 06 22:25:16.023125 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:16.023001    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/27c8bc73-d90a-4cee-b0f4-28c7b6d06012-node-exporter-wtmp\") pod \"node-exporter-gzfm7\" (UID: \"27c8bc73-d90a-4cee-b0f4-28c7b6d06012\") " pod="openshift-monitoring/node-exporter-gzfm7"
Jul 06 22:25:16.023125 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:16.023050    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/27c8bc73-d90a-4cee-b0f4-28c7b6d06012-node-exporter-textfile\") pod \"node-exporter-gzfm7\" (UID: \"27c8bc73-d90a-4cee-b0f4-28c7b6d06012\") " pod="openshift-monitoring/node-exporter-gzfm7"
Jul 06 22:25:16.023125 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:16.023080    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/27c8bc73-d90a-4cee-b0f4-28c7b6d06012-node-exporter-tls\") pod \"node-exporter-gzfm7\" (UID: \"27c8bc73-d90a-4cee-b0f4-28c7b6d06012\") " pod="openshift-monitoring/node-exporter-gzfm7"
Jul 06 22:25:16.023125 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:16.023109    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/27c8bc73-d90a-4cee-b0f4-28c7b6d06012-node-exporter-accelerators-collector-config\") pod \"node-exporter-gzfm7\" (UID: \"27c8bc73-d90a-4cee-b0f4-28c7b6d06012\") " pod="openshift-monitoring/node-exporter-gzfm7"
Jul 06 22:25:16.023767 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:16.023137    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/f098e977-5c44-4797-b931-19ab8d0165a7-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-zldgb\" (UID: \"f098e977-5c44-4797-b931-19ab8d0165a7\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-zldgb"
Jul 06 22:25:16.023767 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:16.023200    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/27c8bc73-d90a-4cee-b0f4-28c7b6d06012-root\") pod \"node-exporter-gzfm7\" (UID: \"27c8bc73-d90a-4cee-b0f4-28c7b6d06012\") " pod="openshift-monitoring/node-exporter-gzfm7"
Jul 06 22:25:16.023767 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:16.023272    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/27c8bc73-d90a-4cee-b0f4-28c7b6d06012-sys\") pod \"node-exporter-gzfm7\" (UID: \"27c8bc73-d90a-4cee-b0f4-28c7b6d06012\") " pod="openshift-monitoring/node-exporter-gzfm7"
Jul 06 22:25:16.023767 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:16.023358    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/27c8bc73-d90a-4cee-b0f4-28c7b6d06012-sys\") pod \"node-exporter-gzfm7\" (UID: \"27c8bc73-d90a-4cee-b0f4-28c7b6d06012\") " pod="openshift-monitoring/node-exporter-gzfm7"
Jul 06 22:25:16.023767 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:16.023369    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/27c8bc73-d90a-4cee-b0f4-28c7b6d06012-metrics-client-ca\") pod \"node-exporter-gzfm7\" (UID: \"27c8bc73-d90a-4cee-b0f4-28c7b6d06012\") " pod="openshift-monitoring/node-exporter-gzfm7"
Jul 06 22:25:16.023767 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:16.023427    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/27c8bc73-d90a-4cee-b0f4-28c7b6d06012-root\") pod \"node-exporter-gzfm7\" (UID: \"27c8bc73-d90a-4cee-b0f4-28c7b6d06012\") " pod="openshift-monitoring/node-exporter-gzfm7"
Jul 06 22:25:16.023767 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:16.023529    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/27c8bc73-d90a-4cee-b0f4-28c7b6d06012-node-exporter-textfile\") pod \"node-exporter-gzfm7\" (UID: \"27c8bc73-d90a-4cee-b0f4-28c7b6d06012\") " pod="openshift-monitoring/node-exporter-gzfm7"
Jul 06 22:25:16.023767 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:16.023693    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/27c8bc73-d90a-4cee-b0f4-28c7b6d06012-node-exporter-accelerators-collector-config\") pod \"node-exporter-gzfm7\" (UID: \"27c8bc73-d90a-4cee-b0f4-28c7b6d06012\") " pod="openshift-monitoring/node-exporter-gzfm7"
Jul 06 22:25:16.024209 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:16.023848    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/27c8bc73-d90a-4cee-b0f4-28c7b6d06012-node-exporter-wtmp\") pod \"node-exporter-gzfm7\" (UID: \"27c8bc73-d90a-4cee-b0f4-28c7b6d06012\") " pod="openshift-monitoring/node-exporter-gzfm7"
Jul 06 22:25:16.025716 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:16.025675    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/27c8bc73-d90a-4cee-b0f4-28c7b6d06012-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-gzfm7\" (UID: \"27c8bc73-d90a-4cee-b0f4-28c7b6d06012\") " pod="openshift-monitoring/node-exporter-gzfm7"
Jul 06 22:25:16.025819 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:16.025806    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/27c8bc73-d90a-4cee-b0f4-28c7b6d06012-node-exporter-tls\") pod \"node-exporter-gzfm7\" (UID: \"27c8bc73-d90a-4cee-b0f4-28c7b6d06012\") " pod="openshift-monitoring/node-exporter-gzfm7"
Jul 06 22:25:16.030939 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:16.030903    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kdqzm\" (UniqueName: \"kubernetes.io/projected/27c8bc73-d90a-4cee-b0f4-28c7b6d06012-kube-api-access-kdqzm\") pod \"node-exporter-gzfm7\" (UID: \"27c8bc73-d90a-4cee-b0f4-28c7b6d06012\") " pod="openshift-monitoring/node-exporter-gzfm7"
Jul 06 22:25:16.124321 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:16.124285    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f098e977-5c44-4797-b931-19ab8d0165a7-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-zldgb\" (UID: \"f098e977-5c44-4797-b931-19ab8d0165a7\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-zldgb"
Jul 06 22:25:16.124321 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:16.124326    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f098e977-5c44-4797-b931-19ab8d0165a7-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-zldgb\" (UID: \"f098e977-5c44-4797-b931-19ab8d0165a7\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-zldgb"
Jul 06 22:25:16.124585 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:16.124357    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/f098e977-5c44-4797-b931-19ab8d0165a7-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-zldgb\" (UID: \"f098e977-5c44-4797-b931-19ab8d0165a7\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-zldgb"
Jul 06 22:25:16.124585 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:16.124382    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fgjdl\" (UniqueName: \"kubernetes.io/projected/f098e977-5c44-4797-b931-19ab8d0165a7-kube-api-access-fgjdl\") pod \"kube-state-metrics-7c9dbb8b7d-zldgb\" (UID: \"f098e977-5c44-4797-b931-19ab8d0165a7\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-zldgb"
Jul 06 22:25:16.124585 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:16.124438    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/f098e977-5c44-4797-b931-19ab8d0165a7-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-zldgb\" (UID: \"f098e977-5c44-4797-b931-19ab8d0165a7\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-zldgb"
Jul 06 22:25:16.124585 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:16.124505    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/f098e977-5c44-4797-b931-19ab8d0165a7-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-zldgb\" (UID: \"f098e977-5c44-4797-b931-19ab8d0165a7\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-zldgb"
Jul 06 22:25:16.124788 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:25:16.124598    2563 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found
Jul 06 22:25:16.124788 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:25:16.124670    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f098e977-5c44-4797-b931-19ab8d0165a7-kube-state-metrics-tls podName:f098e977-5c44-4797-b931-19ab8d0165a7 nodeName:}" failed. No retries permitted until 2026-07-06 22:25:16.624648857 +0000 UTC m=+82.049695379 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/f098e977-5c44-4797-b931-19ab8d0165a7-kube-state-metrics-tls") pod "kube-state-metrics-7c9dbb8b7d-zldgb" (UID: "f098e977-5c44-4797-b931-19ab8d0165a7") : secret "kube-state-metrics-tls" not found
Jul 06 22:25:16.125312 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:16.125283    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f098e977-5c44-4797-b931-19ab8d0165a7-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-zldgb\" (UID: \"f098e977-5c44-4797-b931-19ab8d0165a7\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-zldgb"
Jul 06 22:25:16.125459 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:16.125356    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/f098e977-5c44-4797-b931-19ab8d0165a7-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-zldgb\" (UID: \"f098e977-5c44-4797-b931-19ab8d0165a7\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-zldgb"
Jul 06 22:25:16.125535 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:16.125479    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/f098e977-5c44-4797-b931-19ab8d0165a7-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-zldgb\" (UID: \"f098e977-5c44-4797-b931-19ab8d0165a7\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-zldgb"
Jul 06 22:25:16.127347 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:16.127324    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f098e977-5c44-4797-b931-19ab8d0165a7-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-zldgb\" (UID: \"f098e977-5c44-4797-b931-19ab8d0165a7\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-zldgb"
Jul 06 22:25:16.133996 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:16.133977    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fgjdl\" (UniqueName: \"kubernetes.io/projected/f098e977-5c44-4797-b931-19ab8d0165a7-kube-api-access-fgjdl\") pod \"kube-state-metrics-7c9dbb8b7d-zldgb\" (UID: \"f098e977-5c44-4797-b931-19ab8d0165a7\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-zldgb"
Jul 06 22:25:16.199851 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:16.199820    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-gzfm7"
Jul 06 22:25:16.212435 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:25:16.212401    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27c8bc73_d90a_4cee_b0f4_28c7b6d06012.slice/crio-deea154763476dcf1a0b1a1726b65f29b0eb42733ebdff7533e43f03a0b92a24 WatchSource:0}: Error finding container deea154763476dcf1a0b1a1726b65f29b0eb42733ebdff7533e43f03a0b92a24: Status 404 returned error can't find the container with id deea154763476dcf1a0b1a1726b65f29b0eb42733ebdff7533e43f03a0b92a24
Jul 06 22:25:16.503691 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:16.503600    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-gzfm7" event={"ID":"27c8bc73-d90a-4cee-b0f4-28c7b6d06012","Type":"ContainerStarted","Data":"deea154763476dcf1a0b1a1726b65f29b0eb42733ebdff7533e43f03a0b92a24"}
Jul 06 22:25:16.629532 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:16.629489    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/f098e977-5c44-4797-b931-19ab8d0165a7-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-zldgb\" (UID: \"f098e977-5c44-4797-b931-19ab8d0165a7\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-zldgb"
Jul 06 22:25:16.632490 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:16.632460    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/f098e977-5c44-4797-b931-19ab8d0165a7-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-zldgb\" (UID: \"f098e977-5c44-4797-b931-19ab8d0165a7\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-zldgb"
Jul 06 22:25:16.824112 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:16.824033    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-zldgb"
Jul 06 22:25:16.998812 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:16.998771    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-zldgb"]
Jul 06 22:25:17.002847 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:25:17.002815    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf098e977_5c44_4797_b931_19ab8d0165a7.slice/crio-a1223eee993a712fe64998f245b1e9920ba271b74462aa62ad693779c4eef556 WatchSource:0}: Error finding container a1223eee993a712fe64998f245b1e9920ba271b74462aa62ad693779c4eef556: Status 404 returned error can't find the container with id a1223eee993a712fe64998f245b1e9920ba271b74462aa62ad693779c4eef556
Jul 06 22:25:17.509414 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:17.509382    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-gzfm7" event={"ID":"27c8bc73-d90a-4cee-b0f4-28c7b6d06012","Type":"ContainerStarted","Data":"9ca7638f854ef74adc8d635bfc234c8e9c6f509629e56e12bede8c65da3c2021"}
Jul 06 22:25:17.511515 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:17.511489    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-zldgb" event={"ID":"f098e977-5c44-4797-b931-19ab8d0165a7","Type":"ContainerStarted","Data":"a1223eee993a712fe64998f245b1e9920ba271b74462aa62ad693779c4eef556"}
Jul 06 22:25:17.840807 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:17.840770    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5cb7bf68d4-gb8hs"]
Jul 06 22:25:17.844165 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:17.844135    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5cb7bf68d4-gb8hs"
Jul 06 22:25:17.847242 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:17.847222    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 06 22:25:17.847356 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:17.847271    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 06 22:25:17.847416 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:17.847385    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 06 22:25:17.847594 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:17.847575    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-hcvcf\""
Jul 06 22:25:17.847773 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:17.847759    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 06 22:25:17.847954 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:17.847941    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 06 22:25:17.853958 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:17.853931    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5cb7bf68d4-gb8hs"]
Jul 06 22:25:17.939868 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:17.939836    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d019cfb8-f568-4a0e-be12-45f7557ac2c6-console-serving-cert\") pod \"console-5cb7bf68d4-gb8hs\" (UID: \"d019cfb8-f568-4a0e-be12-45f7557ac2c6\") " pod="openshift-console/console-5cb7bf68d4-gb8hs"
Jul 06 22:25:17.940050 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:17.939874    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d019cfb8-f568-4a0e-be12-45f7557ac2c6-console-oauth-config\") pod \"console-5cb7bf68d4-gb8hs\" (UID: \"d019cfb8-f568-4a0e-be12-45f7557ac2c6\") " pod="openshift-console/console-5cb7bf68d4-gb8hs"
Jul 06 22:25:17.940050 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:17.939899    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d019cfb8-f568-4a0e-be12-45f7557ac2c6-console-config\") pod \"console-5cb7bf68d4-gb8hs\" (UID: \"d019cfb8-f568-4a0e-be12-45f7557ac2c6\") " pod="openshift-console/console-5cb7bf68d4-gb8hs"
Jul 06 22:25:17.940050 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:17.939939    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jqdks\" (UniqueName: \"kubernetes.io/projected/d019cfb8-f568-4a0e-be12-45f7557ac2c6-kube-api-access-jqdks\") pod \"console-5cb7bf68d4-gb8hs\" (UID: \"d019cfb8-f568-4a0e-be12-45f7557ac2c6\") " pod="openshift-console/console-5cb7bf68d4-gb8hs"
Jul 06 22:25:17.940050 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:17.939964    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d019cfb8-f568-4a0e-be12-45f7557ac2c6-service-ca\") pod \"console-5cb7bf68d4-gb8hs\" (UID: \"d019cfb8-f568-4a0e-be12-45f7557ac2c6\") " pod="openshift-console/console-5cb7bf68d4-gb8hs"
Jul 06 22:25:17.940050 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:17.940004    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d019cfb8-f568-4a0e-be12-45f7557ac2c6-oauth-serving-cert\") pod \"console-5cb7bf68d4-gb8hs\" (UID: \"d019cfb8-f568-4a0e-be12-45f7557ac2c6\") " pod="openshift-console/console-5cb7bf68d4-gb8hs"
Jul 06 22:25:18.041263 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:18.041226    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d019cfb8-f568-4a0e-be12-45f7557ac2c6-console-serving-cert\") pod \"console-5cb7bf68d4-gb8hs\" (UID: \"d019cfb8-f568-4a0e-be12-45f7557ac2c6\") " pod="openshift-console/console-5cb7bf68d4-gb8hs"
Jul 06 22:25:18.041681 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:18.041276    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d019cfb8-f568-4a0e-be12-45f7557ac2c6-console-oauth-config\") pod \"console-5cb7bf68d4-gb8hs\" (UID: \"d019cfb8-f568-4a0e-be12-45f7557ac2c6\") " pod="openshift-console/console-5cb7bf68d4-gb8hs"
Jul 06 22:25:18.041681 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:18.041298    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d019cfb8-f568-4a0e-be12-45f7557ac2c6-console-config\") pod \"console-5cb7bf68d4-gb8hs\" (UID: \"d019cfb8-f568-4a0e-be12-45f7557ac2c6\") " pod="openshift-console/console-5cb7bf68d4-gb8hs"
Jul 06 22:25:18.041681 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:18.041334    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jqdks\" (UniqueName: \"kubernetes.io/projected/d019cfb8-f568-4a0e-be12-45f7557ac2c6-kube-api-access-jqdks\") pod \"console-5cb7bf68d4-gb8hs\" (UID: \"d019cfb8-f568-4a0e-be12-45f7557ac2c6\") " pod="openshift-console/console-5cb7bf68d4-gb8hs"
Jul 06 22:25:18.041681 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:18.041365    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d019cfb8-f568-4a0e-be12-45f7557ac2c6-service-ca\") pod \"console-5cb7bf68d4-gb8hs\" (UID: \"d019cfb8-f568-4a0e-be12-45f7557ac2c6\") " pod="openshift-console/console-5cb7bf68d4-gb8hs"
Jul 06 22:25:18.041681 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:18.041402    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d019cfb8-f568-4a0e-be12-45f7557ac2c6-oauth-serving-cert\") pod \"console-5cb7bf68d4-gb8hs\" (UID: \"d019cfb8-f568-4a0e-be12-45f7557ac2c6\") " pod="openshift-console/console-5cb7bf68d4-gb8hs"
Jul 06 22:25:18.042192 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:18.042144    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d019cfb8-f568-4a0e-be12-45f7557ac2c6-oauth-serving-cert\") pod \"console-5cb7bf68d4-gb8hs\" (UID: \"d019cfb8-f568-4a0e-be12-45f7557ac2c6\") " pod="openshift-console/console-5cb7bf68d4-gb8hs"
Jul 06 22:25:18.042517 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:18.042495    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d019cfb8-f568-4a0e-be12-45f7557ac2c6-service-ca\") pod \"console-5cb7bf68d4-gb8hs\" (UID: \"d019cfb8-f568-4a0e-be12-45f7557ac2c6\") " pod="openshift-console/console-5cb7bf68d4-gb8hs"
Jul 06 22:25:18.042698 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:18.042673    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d019cfb8-f568-4a0e-be12-45f7557ac2c6-console-config\") pod \"console-5cb7bf68d4-gb8hs\" (UID: \"d019cfb8-f568-4a0e-be12-45f7557ac2c6\") " pod="openshift-console/console-5cb7bf68d4-gb8hs"
Jul 06 22:25:18.044537 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:18.044518    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d019cfb8-f568-4a0e-be12-45f7557ac2c6-console-oauth-config\") pod \"console-5cb7bf68d4-gb8hs\" (UID: \"d019cfb8-f568-4a0e-be12-45f7557ac2c6\") " pod="openshift-console/console-5cb7bf68d4-gb8hs"
Jul 06 22:25:18.044625 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:18.044588    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d019cfb8-f568-4a0e-be12-45f7557ac2c6-console-serving-cert\") pod \"console-5cb7bf68d4-gb8hs\" (UID: \"d019cfb8-f568-4a0e-be12-45f7557ac2c6\") " pod="openshift-console/console-5cb7bf68d4-gb8hs"
Jul 06 22:25:18.050058 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:18.050039    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jqdks\" (UniqueName: \"kubernetes.io/projected/d019cfb8-f568-4a0e-be12-45f7557ac2c6-kube-api-access-jqdks\") pod \"console-5cb7bf68d4-gb8hs\" (UID: \"d019cfb8-f568-4a0e-be12-45f7557ac2c6\") " pod="openshift-console/console-5cb7bf68d4-gb8hs"
Jul 06 22:25:18.157138 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:18.157033    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5cb7bf68d4-gb8hs"
Jul 06 22:25:18.507450 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:18.507417    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5cb7bf68d4-gb8hs"]
Jul 06 22:25:18.517227 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:18.517192    2563 generic.go:356] "Generic (PLEG): container finished" podID="27c8bc73-d90a-4cee-b0f4-28c7b6d06012" containerID="9ca7638f854ef74adc8d635bfc234c8e9c6f509629e56e12bede8c65da3c2021" exitCode=0
Jul 06 22:25:18.517356 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:18.517286    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-gzfm7" event={"ID":"27c8bc73-d90a-4cee-b0f4-28c7b6d06012","Type":"ContainerDied","Data":"9ca7638f854ef74adc8d635bfc234c8e9c6f509629e56e12bede8c65da3c2021"}
Jul 06 22:25:18.519739 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:18.519716    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-zldgb" event={"ID":"f098e977-5c44-4797-b931-19ab8d0165a7","Type":"ContainerStarted","Data":"9c038ac754dc6f7fc90208cbfd3dea984c25f442fc1611fe4337ede9e105ec57"}
Jul 06 22:25:20.475725 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:20.475698    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-bv5xq"
Jul 06 22:25:20.615346 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:20.615305    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-p4fvh"]
Jul 06 22:25:20.618902 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:20.618880    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-p4fvh"
Jul 06 22:25:20.621664 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:20.621642    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-7vsq6\""
Jul 06 22:25:20.621799 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:20.621673    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Jul 06 22:25:20.629152 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:20.629123    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-p4fvh"]
Jul 06 22:25:20.663071 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:20.663035    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a0f70680-ae20-4ec1-94dd-a29c958a8d53-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-p4fvh\" (UID: \"a0f70680-ae20-4ec1-94dd-a29c958a8d53\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-p4fvh"
Jul 06 22:25:20.663255 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:20.663187    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8bbps\" (UniqueName: \"kubernetes.io/projected/a0f70680-ae20-4ec1-94dd-a29c958a8d53-kube-api-access-8bbps\") pod \"monitoring-plugin-d5dcff7cf-p4fvh\" (UID: \"a0f70680-ae20-4ec1-94dd-a29c958a8d53\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-p4fvh"
Jul 06 22:25:20.764272 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:20.764182    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a0f70680-ae20-4ec1-94dd-a29c958a8d53-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-p4fvh\" (UID: \"a0f70680-ae20-4ec1-94dd-a29c958a8d53\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-p4fvh"
Jul 06 22:25:20.764272 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:20.764271    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8bbps\" (UniqueName: \"kubernetes.io/projected/a0f70680-ae20-4ec1-94dd-a29c958a8d53-kube-api-access-8bbps\") pod \"monitoring-plugin-d5dcff7cf-p4fvh\" (UID: \"a0f70680-ae20-4ec1-94dd-a29c958a8d53\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-p4fvh"
Jul 06 22:25:20.767135 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:20.767105    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a0f70680-ae20-4ec1-94dd-a29c958a8d53-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-p4fvh\" (UID: \"a0f70680-ae20-4ec1-94dd-a29c958a8d53\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-p4fvh"
Jul 06 22:25:20.773804 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:20.773780    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8bbps\" (UniqueName: \"kubernetes.io/projected/a0f70680-ae20-4ec1-94dd-a29c958a8d53-kube-api-access-8bbps\") pod \"monitoring-plugin-d5dcff7cf-p4fvh\" (UID: \"a0f70680-ae20-4ec1-94dd-a29c958a8d53\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-p4fvh"
Jul 06 22:25:20.929372 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:20.929334    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-p4fvh"
Jul 06 22:25:22.110919 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:22.109282    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 06 22:25:22.209632 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:22.209589    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 06 22:25:22.209827 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:22.209722    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 22:25:22.214048 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:22.214010    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 06 22:25:22.214165 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:22.214085    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 06 22:25:22.214248 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:22.214219    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 06 22:25:22.214304 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:22.214244    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 06 22:25:22.214369 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:22.214335    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 06 22:25:22.215229 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:22.215076    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 06 22:25:22.215229 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:22.215116    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 06 22:25:22.215229 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:22.215125    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 06 22:25:22.215462 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:22.215289    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 06 22:25:22.215586 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:22.215566    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 06 22:25:22.215695 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:22.215655    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 06 22:25:22.215695 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:22.215662    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 06 22:25:22.215695 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:22.215686    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-6shnk\""
Jul 06 22:25:22.215866 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:22.215655    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 06 22:25:22.215966 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:22.215947    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-3hmca37nvuqpp\""
Jul 06 22:25:22.218474 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:22.218448    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 06 22:25:22.222277 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:22.222248    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 06 22:25:22.277433 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:22.277402    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/858cf195-514b-452e-9d6a-57605def0790-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"858cf195-514b-452e-9d6a-57605def0790\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 22:25:22.277694 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:22.277445    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/858cf195-514b-452e-9d6a-57605def0790-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"858cf195-514b-452e-9d6a-57605def0790\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 22:25:22.277694 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:22.277473    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/858cf195-514b-452e-9d6a-57605def0790-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"858cf195-514b-452e-9d6a-57605def0790\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 22:25:22.277694 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:22.277514    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/858cf195-514b-452e-9d6a-57605def0790-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"858cf195-514b-452e-9d6a-57605def0790\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 22:25:22.277694 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:22.277564    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/858cf195-514b-452e-9d6a-57605def0790-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"858cf195-514b-452e-9d6a-57605def0790\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 22:25:22.277694 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:22.277610    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/858cf195-514b-452e-9d6a-57605def0790-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"858cf195-514b-452e-9d6a-57605def0790\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 22:25:22.277694 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:22.277633    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/858cf195-514b-452e-9d6a-57605def0790-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"858cf195-514b-452e-9d6a-57605def0790\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 22:25:22.277694 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:22.277652    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/858cf195-514b-452e-9d6a-57605def0790-config-out\") pod \"prometheus-k8s-0\" (UID: \"858cf195-514b-452e-9d6a-57605def0790\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 22:25:22.277694 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:22.277674    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-74z9p\" (UniqueName: \"kubernetes.io/projected/858cf195-514b-452e-9d6a-57605def0790-kube-api-access-74z9p\") pod \"prometheus-k8s-0\" (UID: \"858cf195-514b-452e-9d6a-57605def0790\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 22:25:22.278092 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:22.277714    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/858cf195-514b-452e-9d6a-57605def0790-config\") pod \"prometheus-k8s-0\" (UID: \"858cf195-514b-452e-9d6a-57605def0790\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 22:25:22.278092 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:22.277741    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/858cf195-514b-452e-9d6a-57605def0790-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"858cf195-514b-452e-9d6a-57605def0790\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 22:25:22.278092 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:22.277758    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/858cf195-514b-452e-9d6a-57605def0790-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"858cf195-514b-452e-9d6a-57605def0790\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 22:25:22.278092 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:22.277783    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/858cf195-514b-452e-9d6a-57605def0790-web-config\") pod \"prometheus-k8s-0\" (UID: \"858cf195-514b-452e-9d6a-57605def0790\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 22:25:22.278092 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:22.277803    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/858cf195-514b-452e-9d6a-57605def0790-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"858cf195-514b-452e-9d6a-57605def0790\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 22:25:22.278092 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:22.277822    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/858cf195-514b-452e-9d6a-57605def0790-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"858cf195-514b-452e-9d6a-57605def0790\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 22:25:22.278092 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:22.277886    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/858cf195-514b-452e-9d6a-57605def0790-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"858cf195-514b-452e-9d6a-57605def0790\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 22:25:22.278092 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:22.277920    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/858cf195-514b-452e-9d6a-57605def0790-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"858cf195-514b-452e-9d6a-57605def0790\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 22:25:22.278092 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:22.277946    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/858cf195-514b-452e-9d6a-57605def0790-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"858cf195-514b-452e-9d6a-57605def0790\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 22:25:22.278092 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:22.277975    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/858cf195-514b-452e-9d6a-57605def0790-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"858cf195-514b-452e-9d6a-57605def0790\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 22:25:22.278092 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:22.277996    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/858cf195-514b-452e-9d6a-57605def0790-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"858cf195-514b-452e-9d6a-57605def0790\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 22:25:22.378828 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:22.378733    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/858cf195-514b-452e-9d6a-57605def0790-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"858cf195-514b-452e-9d6a-57605def0790\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 22:25:22.378828 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:22.378794    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/858cf195-514b-452e-9d6a-57605def0790-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"858cf195-514b-452e-9d6a-57605def0790\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 22:25:22.378828 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:22.378819    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/858cf195-514b-452e-9d6a-57605def0790-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"858cf195-514b-452e-9d6a-57605def0790\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 22:25:22.379116 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:22.378838    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/858cf195-514b-452e-9d6a-57605def0790-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"858cf195-514b-452e-9d6a-57605def0790\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 22:25:22.379116 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:22.378856    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/858cf195-514b-452e-9d6a-57605def0790-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"858cf195-514b-452e-9d6a-57605def0790\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 22:25:22.379116 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:22.378887    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/858cf195-514b-452e-9d6a-57605def0790-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"858cf195-514b-452e-9d6a-57605def0790\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 22:25:22.379116 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:22.378906    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/858cf195-514b-452e-9d6a-57605def0790-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"858cf195-514b-452e-9d6a-57605def0790\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 22:25:22.379116 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:22.378929    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/858cf195-514b-452e-9d6a-57605def0790-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"858cf195-514b-452e-9d6a-57605def0790\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 22:25:22.379116 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:22.378965    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/858cf195-514b-452e-9d6a-57605def0790-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"858cf195-514b-452e-9d6a-57605def0790\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 22:25:22.379116 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:22.378979    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/858cf195-514b-452e-9d6a-57605def0790-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"858cf195-514b-452e-9d6a-57605def0790\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 22:25:22.379116 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:22.378994    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/858cf195-514b-452e-9d6a-57605def0790-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"858cf195-514b-452e-9d6a-57605def0790\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 22:25:22.379116 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:22.379010    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/858cf195-514b-452e-9d6a-57605def0790-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"858cf195-514b-452e-9d6a-57605def0790\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 22:25:22.379116 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:22.379028    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/858cf195-514b-452e-9d6a-57605def0790-config-out\") pod \"prometheus-k8s-0\" (UID: \"858cf195-514b-452e-9d6a-57605def0790\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 22:25:22.379116 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:22.379041    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-74z9p\" (UniqueName: \"kubernetes.io/projected/858cf195-514b-452e-9d6a-57605def0790-kube-api-access-74z9p\") pod \"prometheus-k8s-0\" (UID: \"858cf195-514b-452e-9d6a-57605def0790\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 22:25:22.379116 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:22.379061    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/858cf195-514b-452e-9d6a-57605def0790-config\") pod \"prometheus-k8s-0\" (UID: \"858cf195-514b-452e-9d6a-57605def0790\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 22:25:22.379116 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:22.379085    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/858cf195-514b-452e-9d6a-57605def0790-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"858cf195-514b-452e-9d6a-57605def0790\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 22:25:22.379116 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:22.379099    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/858cf195-514b-452e-9d6a-57605def0790-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"858cf195-514b-452e-9d6a-57605def0790\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 22:25:22.379116 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:22.379114    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/858cf195-514b-452e-9d6a-57605def0790-web-config\") pod \"prometheus-k8s-0\" (UID: \"858cf195-514b-452e-9d6a-57605def0790\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 22:25:22.379880 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:22.379133    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/858cf195-514b-452e-9d6a-57605def0790-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"858cf195-514b-452e-9d6a-57605def0790\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 22:25:22.379880 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:22.379151    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/858cf195-514b-452e-9d6a-57605def0790-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"858cf195-514b-452e-9d6a-57605def0790\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 22:25:22.379880 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:22.379585    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/858cf195-514b-452e-9d6a-57605def0790-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"858cf195-514b-452e-9d6a-57605def0790\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 22:25:22.379880 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:22.379722    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/858cf195-514b-452e-9d6a-57605def0790-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"858cf195-514b-452e-9d6a-57605def0790\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 22:25:22.379880 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:22.379806    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/858cf195-514b-452e-9d6a-57605def0790-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"858cf195-514b-452e-9d6a-57605def0790\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 22:25:22.380757 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:22.380723    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/858cf195-514b-452e-9d6a-57605def0790-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"858cf195-514b-452e-9d6a-57605def0790\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 22:25:22.383117 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:22.382779    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/858cf195-514b-452e-9d6a-57605def0790-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"858cf195-514b-452e-9d6a-57605def0790\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 22:25:22.383117 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:22.382804    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/858cf195-514b-452e-9d6a-57605def0790-config-out\") pod \"prometheus-k8s-0\" (UID: \"858cf195-514b-452e-9d6a-57605def0790\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 22:25:22.383117 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:22.382892    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/858cf195-514b-452e-9d6a-57605def0790-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"858cf195-514b-452e-9d6a-57605def0790\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 22:25:22.383117 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:22.382975    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/858cf195-514b-452e-9d6a-57605def0790-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"858cf195-514b-452e-9d6a-57605def0790\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 22:25:22.383477 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:22.383382    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/858cf195-514b-452e-9d6a-57605def0790-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"858cf195-514b-452e-9d6a-57605def0790\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 22:25:22.383679 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:22.383625    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/858cf195-514b-452e-9d6a-57605def0790-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"858cf195-514b-452e-9d6a-57605def0790\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 22:25:22.384354 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:22.384005    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/858cf195-514b-452e-9d6a-57605def0790-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"858cf195-514b-452e-9d6a-57605def0790\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 22:25:22.384729 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:22.384650    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/858cf195-514b-452e-9d6a-57605def0790-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"858cf195-514b-452e-9d6a-57605def0790\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 22:25:22.385957 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:22.385934    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/858cf195-514b-452e-9d6a-57605def0790-web-config\") pod \"prometheus-k8s-0\" (UID: \"858cf195-514b-452e-9d6a-57605def0790\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 22:25:22.386068 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:22.385982    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/858cf195-514b-452e-9d6a-57605def0790-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"858cf195-514b-452e-9d6a-57605def0790\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 22:25:22.386474 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:22.386418    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/858cf195-514b-452e-9d6a-57605def0790-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"858cf195-514b-452e-9d6a-57605def0790\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 22:25:22.388364 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:22.388294    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/858cf195-514b-452e-9d6a-57605def0790-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"858cf195-514b-452e-9d6a-57605def0790\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 22:25:22.388364 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:22.388309    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-74z9p\" (UniqueName: \"kubernetes.io/projected/858cf195-514b-452e-9d6a-57605def0790-kube-api-access-74z9p\") pod \"prometheus-k8s-0\" (UID: \"858cf195-514b-452e-9d6a-57605def0790\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 22:25:22.388525 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:22.388371    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/858cf195-514b-452e-9d6a-57605def0790-config\") pod \"prometheus-k8s-0\" (UID: \"858cf195-514b-452e-9d6a-57605def0790\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 22:25:22.388802 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:22.388775    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/858cf195-514b-452e-9d6a-57605def0790-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"858cf195-514b-452e-9d6a-57605def0790\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 22:25:22.389385 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:22.389346    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/858cf195-514b-452e-9d6a-57605def0790-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"858cf195-514b-452e-9d6a-57605def0790\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 22:25:22.523618 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:22.523579    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 22:25:25.848040 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:25:25.848001    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd019cfb8_f568_4a0e_be12_45f7557ac2c6.slice/crio-351673a6c6134827bce2c1f0f74c4f91964995830db41c2ac155b19779c059c9 WatchSource:0}: Error finding container 351673a6c6134827bce2c1f0f74c4f91964995830db41c2ac155b19779c059c9: Status 404 returned error can't find the container with id 351673a6c6134827bce2c1f0f74c4f91964995830db41c2ac155b19779c059c9
Jul 06 22:25:26.025500 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:26.025373    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 06 22:25:26.037362 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:26.037265    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-p4fvh"]
Jul 06 22:25:26.058076 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:25:26.057998    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod858cf195_514b_452e_9d6a_57605def0790.slice/crio-c218f60f9921e51c82d6a023ce3d931d9e62e7690a0185e809e1a5f19f008018 WatchSource:0}: Error finding container c218f60f9921e51c82d6a023ce3d931d9e62e7690a0185e809e1a5f19f008018: Status 404 returned error can't find the container with id c218f60f9921e51c82d6a023ce3d931d9e62e7690a0185e809e1a5f19f008018
Jul 06 22:25:26.059118 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:25:26.059092    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0f70680_ae20_4ec1_94dd_a29c958a8d53.slice/crio-88b84d96969791c15202fd7922c0481619160e4a3d4efafcc296573c0cf4f810 WatchSource:0}: Error finding container 88b84d96969791c15202fd7922c0481619160e4a3d4efafcc296573c0cf4f810: Status 404 returned error can't find the container with id 88b84d96969791c15202fd7922c0481619160e4a3d4efafcc296573c0cf4f810
Jul 06 22:25:26.559643 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:26.559591    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"858cf195-514b-452e-9d6a-57605def0790","Type":"ContainerStarted","Data":"c218f60f9921e51c82d6a023ce3d931d9e62e7690a0185e809e1a5f19f008018"}
Jul 06 22:25:26.563920 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:26.563881    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-zldgb" event={"ID":"f098e977-5c44-4797-b931-19ab8d0165a7","Type":"ContainerStarted","Data":"5410aab4708b080690d0b51cae2a2907162c5251fac70b089737e1032e11effc"}
Jul 06 22:25:26.564069 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:26.563978    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-zldgb" event={"ID":"f098e977-5c44-4797-b931-19ab8d0165a7","Type":"ContainerStarted","Data":"216e5f38fc0b890b5db39169b768724f3bd64f83ed45a9764847e688723e5009"}
Jul 06 22:25:26.568550 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:26.568463    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5cb7bf68d4-gb8hs" event={"ID":"d019cfb8-f568-4a0e-be12-45f7557ac2c6","Type":"ContainerStarted","Data":"351673a6c6134827bce2c1f0f74c4f91964995830db41c2ac155b19779c059c9"}
Jul 06 22:25:26.578468 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:26.578407    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-p4fvh" event={"ID":"a0f70680-ae20-4ec1-94dd-a29c958a8d53","Type":"ContainerStarted","Data":"88b84d96969791c15202fd7922c0481619160e4a3d4efafcc296573c0cf4f810"}
Jul 06 22:25:26.583708 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:26.583638    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-685bfb5f64-lgzh4" event={"ID":"11551149-01c0-4656-be37-d24965aac4a4","Type":"ContainerStarted","Data":"72f2dddb34e1b98c4e748e437eb11e51d49a87750cb8fc83630aa002cf6fbb6c"}
Jul 06 22:25:26.584872 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:26.584852    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-685bfb5f64-lgzh4"
Jul 06 22:25:26.588880 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:26.588856    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-gzfm7" event={"ID":"27c8bc73-d90a-4cee-b0f4-28c7b6d06012","Type":"ContainerStarted","Data":"2c5c4f484ff439a9de2d321cb45cd5154bab66376b5a0c7d00723843e30fcc67"}
Jul 06 22:25:26.589075 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:26.589059    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-gzfm7" event={"ID":"27c8bc73-d90a-4cee-b0f4-28c7b6d06012","Type":"ContainerStarted","Data":"d17b7deee934321863f5e2712b1b32e6d4d3e658d73af528ba88dc4d497bf916"}
Jul 06 22:25:26.593328 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:26.593284    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-zldgb" podStartSLOduration=10.189650278 podStartE2EDuration="11.593247659s" totalImagesPullingTime="1.403597381s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:25:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:25:26.591072748 +0000 UTC m=+92.016119281" watchObservedRunningTime="2026-07-06 22:25:26.593247659 +0000 UTC m=+92.018294213"
Jul 06 22:25:26.599853 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:26.599790    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-685bfb5f64-lgzh4"
Jul 06 22:25:26.609057 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:26.607885    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-685bfb5f64-lgzh4" podStartSLOduration=1.461862327 podStartE2EDuration="19.60786629s" totalImagesPullingTime="18.146003963s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:25:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:25:26.606702854 +0000 UTC m=+92.031749395" watchObservedRunningTime="2026-07-06 22:25:26.60786629 +0000 UTC m=+92.032912832"
Jul 06 22:25:26.632376 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:26.631376    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-gzfm7" podStartSLOduration=10.329810115 podStartE2EDuration="11.631357546s" totalImagesPullingTime="1.216945969s" totalInitContainerRuntime="84.601462ms" isStatefulPod=true podCreationTimestamp="2026-07-06 22:25:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:25:26.63034729 +0000 UTC m=+92.055393892" watchObservedRunningTime="2026-07-06 22:25:26.631357546 +0000 UTC m=+92.056404088"
Jul 06 22:25:30.476792 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:30.476748    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-745f45d84c-z5h56"
Jul 06 22:25:31.616146 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:31.615799    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-p4fvh" event={"ID":"a0f70680-ae20-4ec1-94dd-a29c958a8d53","Type":"ContainerStarted","Data":"4834294c24e2f526c4ddb52feb9cf1d49e0f28a966789291acf441c14f38bbf6"}
Jul 06 22:25:31.616146 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:31.615876    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-p4fvh"
Jul 06 22:25:31.618368 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:31.618332    2563 generic.go:356] "Generic (PLEG): container finished" podID="858cf195-514b-452e-9d6a-57605def0790" containerID="8dc4771a516d0840c7d49ae6c70d1994ffd6b8cb38ccbcbc2aa4a6e66cc39228" exitCode=0
Jul 06 22:25:31.618505 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:31.618389    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"858cf195-514b-452e-9d6a-57605def0790","Type":"ContainerDied","Data":"8dc4771a516d0840c7d49ae6c70d1994ffd6b8cb38ccbcbc2aa4a6e66cc39228"}
Jul 06 22:25:31.620538 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:31.620506    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5cb7bf68d4-gb8hs" event={"ID":"d019cfb8-f568-4a0e-be12-45f7557ac2c6","Type":"ContainerStarted","Data":"10e3516562d41940a3c2d45b935511f799a84f1a06bbc8b426b44fe67e287aa4"}
Jul 06 22:25:31.621984 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:31.621967    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-p4fvh"
Jul 06 22:25:31.633456 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:31.633409    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-p4fvh" podStartSLOduration=7.065319124 podStartE2EDuration="11.633394758s" totalImagesPullingTime="4.568075634s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:25:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:25:31.633114734 +0000 UTC m=+97.058161279" watchObservedRunningTime="2026-07-06 22:25:31.633394758 +0000 UTC m=+97.058441300"
Jul 06 22:25:31.703455 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:31.703395    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5cb7bf68d4-gb8hs" podStartSLOduration=9.906501712 podStartE2EDuration="14.703373636s" totalImagesPullingTime="4.796871924s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:25:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:25:31.701798586 +0000 UTC m=+97.126845127" watchObservedRunningTime="2026-07-06 22:25:31.703373636 +0000 UTC m=+97.128420242"
Jul 06 22:25:36.648991 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:36.648909    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"858cf195-514b-452e-9d6a-57605def0790","Type":"ContainerStarted","Data":"caed834d4d1a297ab56e13de545d129aaaea1b58d1af37e019949ee91705ccf3"}
Jul 06 22:25:36.648991 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:36.648973    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"858cf195-514b-452e-9d6a-57605def0790","Type":"ContainerStarted","Data":"acf16978c8aac2bd1a48745d6d061e11bd3ff66a5766924fd30a3fa0f7c2161c"}
Jul 06 22:25:37.682990 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:37.682943    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5cb7bf68d4-gb8hs"]
Jul 06 22:25:38.157865 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:38.157828    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5cb7bf68d4-gb8hs"
Jul 06 22:25:38.454802 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:38.454771    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-6qmpm"
Jul 06 22:25:39.663522 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:39.663484    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"858cf195-514b-452e-9d6a-57605def0790","Type":"ContainerStarted","Data":"fe64a34ff1ef93e79f33f8efce518d20c05eff9edd2c3b0a85736d9d64c519ed"}
Jul 06 22:25:39.663962 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:39.663617    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"858cf195-514b-452e-9d6a-57605def0790","Type":"ContainerStarted","Data":"50d6658df0f9380712776d280a0f734937e525cc0d271a31088516bcc8f87798"}
Jul 06 22:25:39.663962 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:39.663666    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"858cf195-514b-452e-9d6a-57605def0790","Type":"ContainerStarted","Data":"3c4da65ff69af3667d1afb76aad721ec956224e9d1e8e53929a7852d5b3a24a0"}
Jul 06 22:25:39.663962 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:39.663693    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"858cf195-514b-452e-9d6a-57605def0790","Type":"ContainerStarted","Data":"423fdbf76002e11fda332a3bcf34104b6ee69f100484d4a09ed7de3b7cfa292a"}
Jul 06 22:25:39.693673 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:39.693617    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=6.397900292 podStartE2EDuration="17.693598106s" totalImagesPullingTime="11.260920365s" totalInitContainerRuntime="34.777449ms" isStatefulPod=false podCreationTimestamp="2026-07-06 22:25:22 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:25:39.691613354 +0000 UTC m=+105.116659912" watchObservedRunningTime="2026-07-06 22:25:39.693598106 +0000 UTC m=+105.118644647"
Jul 06 22:25:42.524114 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:25:42.524077    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 22:26:02.705445 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:02.705387    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-5cb7bf68d4-gb8hs" podUID="d019cfb8-f568-4a0e-be12-45f7557ac2c6" containerName="console" containerID="cri-o://10e3516562d41940a3c2d45b935511f799a84f1a06bbc8b426b44fe67e287aa4" gracePeriod=15
Jul 06 22:26:02.988131 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:02.988105    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5cb7bf68d4-gb8hs"
Jul 06 22:26:03.016389 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:03.016361    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d019cfb8-f568-4a0e-be12-45f7557ac2c6-console-config\") pod \"d019cfb8-f568-4a0e-be12-45f7557ac2c6\" (UID: \"d019cfb8-f568-4a0e-be12-45f7557ac2c6\") "
Jul 06 22:26:03.016516 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:03.016401    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d019cfb8-f568-4a0e-be12-45f7557ac2c6-console-serving-cert\") pod \"d019cfb8-f568-4a0e-be12-45f7557ac2c6\" (UID: \"d019cfb8-f568-4a0e-be12-45f7557ac2c6\") "
Jul 06 22:26:03.016516 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:03.016438    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jqdks\" (UniqueName: \"kubernetes.io/projected/d019cfb8-f568-4a0e-be12-45f7557ac2c6-kube-api-access-jqdks\") pod \"d019cfb8-f568-4a0e-be12-45f7557ac2c6\" (UID: \"d019cfb8-f568-4a0e-be12-45f7557ac2c6\") "
Jul 06 22:26:03.016516 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:03.016503    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d019cfb8-f568-4a0e-be12-45f7557ac2c6-console-oauth-config\") pod \"d019cfb8-f568-4a0e-be12-45f7557ac2c6\" (UID: \"d019cfb8-f568-4a0e-be12-45f7557ac2c6\") "
Jul 06 22:26:03.016669 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:03.016573    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d019cfb8-f568-4a0e-be12-45f7557ac2c6-service-ca\") pod \"d019cfb8-f568-4a0e-be12-45f7557ac2c6\" (UID: \"d019cfb8-f568-4a0e-be12-45f7557ac2c6\") "
Jul 06 22:26:03.016669 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:03.016649    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d019cfb8-f568-4a0e-be12-45f7557ac2c6-oauth-serving-cert\") pod \"d019cfb8-f568-4a0e-be12-45f7557ac2c6\" (UID: \"d019cfb8-f568-4a0e-be12-45f7557ac2c6\") "
Jul 06 22:26:03.016793 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:03.016771    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d019cfb8-f568-4a0e-be12-45f7557ac2c6-console-config" (OuterVolumeSpecName: "console-config") pod "d019cfb8-f568-4a0e-be12-45f7557ac2c6" (UID: "d019cfb8-f568-4a0e-be12-45f7557ac2c6"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 22:26:03.016940 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:03.016918    2563 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d019cfb8-f568-4a0e-be12-45f7557ac2c6-console-config\") on node \"ip-10-0-133-105.ec2.internal\" DevicePath \"\""
Jul 06 22:26:03.017046 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:03.016989    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d019cfb8-f568-4a0e-be12-45f7557ac2c6-service-ca" (OuterVolumeSpecName: "service-ca") pod "d019cfb8-f568-4a0e-be12-45f7557ac2c6" (UID: "d019cfb8-f568-4a0e-be12-45f7557ac2c6"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 22:26:03.017368 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:03.017345    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d019cfb8-f568-4a0e-be12-45f7557ac2c6-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "d019cfb8-f568-4a0e-be12-45f7557ac2c6" (UID: "d019cfb8-f568-4a0e-be12-45f7557ac2c6"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 22:26:03.018824 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:03.018797    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d019cfb8-f568-4a0e-be12-45f7557ac2c6-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "d019cfb8-f568-4a0e-be12-45f7557ac2c6" (UID: "d019cfb8-f568-4a0e-be12-45f7557ac2c6"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 22:26:03.018913 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:03.018833    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d019cfb8-f568-4a0e-be12-45f7557ac2c6-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "d019cfb8-f568-4a0e-be12-45f7557ac2c6" (UID: "d019cfb8-f568-4a0e-be12-45f7557ac2c6"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 22:26:03.019057 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:03.019040    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d019cfb8-f568-4a0e-be12-45f7557ac2c6-kube-api-access-jqdks" (OuterVolumeSpecName: "kube-api-access-jqdks") pod "d019cfb8-f568-4a0e-be12-45f7557ac2c6" (UID: "d019cfb8-f568-4a0e-be12-45f7557ac2c6"). InnerVolumeSpecName "kube-api-access-jqdks". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 22:26:03.117711 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:03.117680    2563 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d019cfb8-f568-4a0e-be12-45f7557ac2c6-console-serving-cert\") on node \"ip-10-0-133-105.ec2.internal\" DevicePath \"\""
Jul 06 22:26:03.117711 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:03.117702    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jqdks\" (UniqueName: \"kubernetes.io/projected/d019cfb8-f568-4a0e-be12-45f7557ac2c6-kube-api-access-jqdks\") on node \"ip-10-0-133-105.ec2.internal\" DevicePath \"\""
Jul 06 22:26:03.117711 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:03.117711    2563 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d019cfb8-f568-4a0e-be12-45f7557ac2c6-console-oauth-config\") on node \"ip-10-0-133-105.ec2.internal\" DevicePath \"\""
Jul 06 22:26:03.117961 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:03.117721    2563 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d019cfb8-f568-4a0e-be12-45f7557ac2c6-service-ca\") on node \"ip-10-0-133-105.ec2.internal\" DevicePath \"\""
Jul 06 22:26:03.117961 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:03.117729    2563 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d019cfb8-f568-4a0e-be12-45f7557ac2c6-oauth-serving-cert\") on node \"ip-10-0-133-105.ec2.internal\" DevicePath \"\""
Jul 06 22:26:03.760559 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:03.760525    2563 generic.go:356] "Generic (PLEG): container finished" podID="d019cfb8-f568-4a0e-be12-45f7557ac2c6" containerID="10e3516562d41940a3c2d45b935511f799a84f1a06bbc8b426b44fe67e287aa4" exitCode=0
Jul 06 22:26:03.761011 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:03.760583    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5cb7bf68d4-gb8hs"
Jul 06 22:26:03.761011 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:03.760591    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5cb7bf68d4-gb8hs" event={"ID":"d019cfb8-f568-4a0e-be12-45f7557ac2c6","Type":"ContainerDied","Data":"10e3516562d41940a3c2d45b935511f799a84f1a06bbc8b426b44fe67e287aa4"}
Jul 06 22:26:03.761011 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:03.760630    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5cb7bf68d4-gb8hs" event={"ID":"d019cfb8-f568-4a0e-be12-45f7557ac2c6","Type":"ContainerDied","Data":"351673a6c6134827bce2c1f0f74c4f91964995830db41c2ac155b19779c059c9"}
Jul 06 22:26:03.761011 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:03.760651    2563 scope.go:117] "RemoveContainer" containerID="10e3516562d41940a3c2d45b935511f799a84f1a06bbc8b426b44fe67e287aa4"
Jul 06 22:26:03.780573 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:03.780550    2563 scope.go:117] "RemoveContainer" containerID="10e3516562d41940a3c2d45b935511f799a84f1a06bbc8b426b44fe67e287aa4"
Jul 06 22:26:03.780897 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:26:03.780860    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"10e3516562d41940a3c2d45b935511f799a84f1a06bbc8b426b44fe67e287aa4\": container with ID starting with 10e3516562d41940a3c2d45b935511f799a84f1a06bbc8b426b44fe67e287aa4 not found: ID does not exist" containerID="10e3516562d41940a3c2d45b935511f799a84f1a06bbc8b426b44fe67e287aa4"
Jul 06 22:26:03.780969 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:03.780895    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"10e3516562d41940a3c2d45b935511f799a84f1a06bbc8b426b44fe67e287aa4"} err="failed to get container status \"10e3516562d41940a3c2d45b935511f799a84f1a06bbc8b426b44fe67e287aa4\": rpc error: code = NotFound desc = could not find container \"10e3516562d41940a3c2d45b935511f799a84f1a06bbc8b426b44fe67e287aa4\": container with ID starting with 10e3516562d41940a3c2d45b935511f799a84f1a06bbc8b426b44fe67e287aa4 not found: ID does not exist"
Jul 06 22:26:03.787280 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:03.787253    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5cb7bf68d4-gb8hs"]
Jul 06 22:26:03.790049 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:03.790032    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5cb7bf68d4-gb8hs"]
Jul 06 22:26:05.079681 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:05.079642    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d019cfb8-f568-4a0e-be12-45f7557ac2c6" path="/var/lib/kubelet/pods/d019cfb8-f568-4a0e-be12-45f7557ac2c6/volumes"
Jul 06 22:26:22.524605 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:22.524552    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 22:26:22.545451 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:22.545428    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 22:26:22.855145 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:22.855068    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 22:26:40.417837 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.417804    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 06 22:26:40.418395 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.418093    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="858cf195-514b-452e-9d6a-57605def0790" containerName="prometheus" containerID="cri-o://acf16978c8aac2bd1a48745d6d061e11bd3ff66a5766924fd30a3fa0f7c2161c" gracePeriod=600
Jul 06 22:26:40.418395 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.418133    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="858cf195-514b-452e-9d6a-57605def0790" containerName="thanos-sidecar" containerID="cri-o://423fdbf76002e11fda332a3bcf34104b6ee69f100484d4a09ed7de3b7cfa292a" gracePeriod=600
Jul 06 22:26:40.418395 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.418156    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="858cf195-514b-452e-9d6a-57605def0790" containerName="kube-rbac-proxy-thanos" containerID="cri-o://fe64a34ff1ef93e79f33f8efce518d20c05eff9edd2c3b0a85736d9d64c519ed" gracePeriod=600
Jul 06 22:26:40.418395 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.418205    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="858cf195-514b-452e-9d6a-57605def0790" containerName="kube-rbac-proxy-web" containerID="cri-o://3c4da65ff69af3667d1afb76aad721ec956224e9d1e8e53929a7852d5b3a24a0" gracePeriod=600
Jul 06 22:26:40.418395 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.418133    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="858cf195-514b-452e-9d6a-57605def0790" containerName="kube-rbac-proxy" containerID="cri-o://50d6658df0f9380712776d280a0f734937e525cc0d271a31088516bcc8f87798" gracePeriod=600
Jul 06 22:26:40.418395 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.418362    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="858cf195-514b-452e-9d6a-57605def0790" containerName="config-reloader" containerID="cri-o://caed834d4d1a297ab56e13de545d129aaaea1b58d1af37e019949ee91705ccf3" gracePeriod=600
Jul 06 22:26:40.676641 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.676571    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 22:26:40.818733 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.818701    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/858cf195-514b-452e-9d6a-57605def0790-configmap-kubelet-serving-ca-bundle\") pod \"858cf195-514b-452e-9d6a-57605def0790\" (UID: \"858cf195-514b-452e-9d6a-57605def0790\") "
Jul 06 22:26:40.818733 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.818736    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/858cf195-514b-452e-9d6a-57605def0790-prometheus-k8s-rulefiles-0\") pod \"858cf195-514b-452e-9d6a-57605def0790\" (UID: \"858cf195-514b-452e-9d6a-57605def0790\") "
Jul 06 22:26:40.818938 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.818756    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/858cf195-514b-452e-9d6a-57605def0790-prometheus-k8s-rulefiles-2\") pod \"858cf195-514b-452e-9d6a-57605def0790\" (UID: \"858cf195-514b-452e-9d6a-57605def0790\") "
Jul 06 22:26:40.818938 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.818780    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/858cf195-514b-452e-9d6a-57605def0790-configmap-metrics-client-ca\") pod \"858cf195-514b-452e-9d6a-57605def0790\" (UID: \"858cf195-514b-452e-9d6a-57605def0790\") "
Jul 06 22:26:40.818938 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.818805    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/858cf195-514b-452e-9d6a-57605def0790-config\") pod \"858cf195-514b-452e-9d6a-57605def0790\" (UID: \"858cf195-514b-452e-9d6a-57605def0790\") "
Jul 06 22:26:40.819075 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.818932    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/858cf195-514b-452e-9d6a-57605def0790-web-config\") pod \"858cf195-514b-452e-9d6a-57605def0790\" (UID: \"858cf195-514b-452e-9d6a-57605def0790\") "
Jul 06 22:26:40.819075 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.818980    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/858cf195-514b-452e-9d6a-57605def0790-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"858cf195-514b-452e-9d6a-57605def0790\" (UID: \"858cf195-514b-452e-9d6a-57605def0790\") "
Jul 06 22:26:40.819075 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.819016    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-74z9p\" (UniqueName: \"kubernetes.io/projected/858cf195-514b-452e-9d6a-57605def0790-kube-api-access-74z9p\") pod \"858cf195-514b-452e-9d6a-57605def0790\" (UID: \"858cf195-514b-452e-9d6a-57605def0790\") "
Jul 06 22:26:40.819075 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.819060    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/858cf195-514b-452e-9d6a-57605def0790-thanos-prometheus-http-client-file\") pod \"858cf195-514b-452e-9d6a-57605def0790\" (UID: \"858cf195-514b-452e-9d6a-57605def0790\") "
Jul 06 22:26:40.819309 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.819102    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/858cf195-514b-452e-9d6a-57605def0790-secret-prometheus-k8s-tls\") pod \"858cf195-514b-452e-9d6a-57605def0790\" (UID: \"858cf195-514b-452e-9d6a-57605def0790\") "
Jul 06 22:26:40.819309 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.819126    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/858cf195-514b-452e-9d6a-57605def0790-secret-kube-rbac-proxy\") pod \"858cf195-514b-452e-9d6a-57605def0790\" (UID: \"858cf195-514b-452e-9d6a-57605def0790\") "
Jul 06 22:26:40.819309 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.819154    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/858cf195-514b-452e-9d6a-57605def0790-prometheus-k8s-rulefiles-1\") pod \"858cf195-514b-452e-9d6a-57605def0790\" (UID: \"858cf195-514b-452e-9d6a-57605def0790\") "
Jul 06 22:26:40.819309 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.819239    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/858cf195-514b-452e-9d6a-57605def0790-prometheus-trusted-ca-bundle\") pod \"858cf195-514b-452e-9d6a-57605def0790\" (UID: \"858cf195-514b-452e-9d6a-57605def0790\") "
Jul 06 22:26:40.819309 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.819272    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/858cf195-514b-452e-9d6a-57605def0790-secret-grpc-tls\") pod \"858cf195-514b-452e-9d6a-57605def0790\" (UID: \"858cf195-514b-452e-9d6a-57605def0790\") "
Jul 06 22:26:40.819309 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.819307    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/858cf195-514b-452e-9d6a-57605def0790-secret-metrics-client-certs\") pod \"858cf195-514b-452e-9d6a-57605def0790\" (UID: \"858cf195-514b-452e-9d6a-57605def0790\") "
Jul 06 22:26:40.819606 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.819338    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/858cf195-514b-452e-9d6a-57605def0790-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"858cf195-514b-452e-9d6a-57605def0790\" (UID: \"858cf195-514b-452e-9d6a-57605def0790\") "
Jul 06 22:26:40.819606 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.819369    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/858cf195-514b-452e-9d6a-57605def0790-prometheus-k8s-db\") pod \"858cf195-514b-452e-9d6a-57605def0790\" (UID: \"858cf195-514b-452e-9d6a-57605def0790\") "
Jul 06 22:26:40.819606 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.819416    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/858cf195-514b-452e-9d6a-57605def0790-config-out\") pod \"858cf195-514b-452e-9d6a-57605def0790\" (UID: \"858cf195-514b-452e-9d6a-57605def0790\") "
Jul 06 22:26:40.819606 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.819450    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/858cf195-514b-452e-9d6a-57605def0790-configmap-serving-certs-ca-bundle\") pod \"858cf195-514b-452e-9d6a-57605def0790\" (UID: \"858cf195-514b-452e-9d6a-57605def0790\") "
Jul 06 22:26:40.819606 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.819485    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/858cf195-514b-452e-9d6a-57605def0790-tls-assets\") pod \"858cf195-514b-452e-9d6a-57605def0790\" (UID: \"858cf195-514b-452e-9d6a-57605def0790\") "
Jul 06 22:26:40.821591 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.819148    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/858cf195-514b-452e-9d6a-57605def0790-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "858cf195-514b-452e-9d6a-57605def0790" (UID: "858cf195-514b-452e-9d6a-57605def0790"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 22:26:40.821591 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.819265    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/858cf195-514b-452e-9d6a-57605def0790-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "858cf195-514b-452e-9d6a-57605def0790" (UID: "858cf195-514b-452e-9d6a-57605def0790"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 22:26:40.821591 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.819616    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/858cf195-514b-452e-9d6a-57605def0790-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "858cf195-514b-452e-9d6a-57605def0790" (UID: "858cf195-514b-452e-9d6a-57605def0790"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 22:26:40.821591 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.820112    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/858cf195-514b-452e-9d6a-57605def0790-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "858cf195-514b-452e-9d6a-57605def0790" (UID: "858cf195-514b-452e-9d6a-57605def0790"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 22:26:40.821591 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.821491    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/858cf195-514b-452e-9d6a-57605def0790-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "858cf195-514b-452e-9d6a-57605def0790" (UID: "858cf195-514b-452e-9d6a-57605def0790"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 22:26:40.821934 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.821755    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/858cf195-514b-452e-9d6a-57605def0790-config" (OuterVolumeSpecName: "config") pod "858cf195-514b-452e-9d6a-57605def0790" (UID: "858cf195-514b-452e-9d6a-57605def0790"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 22:26:40.822665 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.822404    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/858cf195-514b-452e-9d6a-57605def0790-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "858cf195-514b-452e-9d6a-57605def0790" (UID: "858cf195-514b-452e-9d6a-57605def0790"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 22:26:40.822665 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.822564    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/858cf195-514b-452e-9d6a-57605def0790-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "858cf195-514b-452e-9d6a-57605def0790" (UID: "858cf195-514b-452e-9d6a-57605def0790"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 22:26:40.822898 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.822819    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/858cf195-514b-452e-9d6a-57605def0790-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "858cf195-514b-452e-9d6a-57605def0790" (UID: "858cf195-514b-452e-9d6a-57605def0790"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 22:26:40.823144 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.823092    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/858cf195-514b-452e-9d6a-57605def0790-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "858cf195-514b-452e-9d6a-57605def0790" (UID: "858cf195-514b-452e-9d6a-57605def0790"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 22:26:40.823227 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.823136    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/858cf195-514b-452e-9d6a-57605def0790-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "858cf195-514b-452e-9d6a-57605def0790" (UID: "858cf195-514b-452e-9d6a-57605def0790"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 22:26:40.823227 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.823196    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/858cf195-514b-452e-9d6a-57605def0790-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "858cf195-514b-452e-9d6a-57605def0790" (UID: "858cf195-514b-452e-9d6a-57605def0790"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 22:26:40.823340 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.823255    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/858cf195-514b-452e-9d6a-57605def0790-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "858cf195-514b-452e-9d6a-57605def0790" (UID: "858cf195-514b-452e-9d6a-57605def0790"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 22:26:40.823975 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.823938    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/858cf195-514b-452e-9d6a-57605def0790-kube-api-access-74z9p" (OuterVolumeSpecName: "kube-api-access-74z9p") pod "858cf195-514b-452e-9d6a-57605def0790" (UID: "858cf195-514b-452e-9d6a-57605def0790"). InnerVolumeSpecName "kube-api-access-74z9p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 22:26:40.824339 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.824316    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/858cf195-514b-452e-9d6a-57605def0790-config-out" (OuterVolumeSpecName: "config-out") pod "858cf195-514b-452e-9d6a-57605def0790" (UID: "858cf195-514b-452e-9d6a-57605def0790"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 22:26:40.824469 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.824448    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/858cf195-514b-452e-9d6a-57605def0790-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "858cf195-514b-452e-9d6a-57605def0790" (UID: "858cf195-514b-452e-9d6a-57605def0790"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 22:26:40.824603 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.824587    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/858cf195-514b-452e-9d6a-57605def0790-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "858cf195-514b-452e-9d6a-57605def0790" (UID: "858cf195-514b-452e-9d6a-57605def0790"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 22:26:40.824935 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.824916    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/858cf195-514b-452e-9d6a-57605def0790-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "858cf195-514b-452e-9d6a-57605def0790" (UID: "858cf195-514b-452e-9d6a-57605def0790"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 22:26:40.825326 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.825300    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/858cf195-514b-452e-9d6a-57605def0790-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "858cf195-514b-452e-9d6a-57605def0790" (UID: "858cf195-514b-452e-9d6a-57605def0790"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 22:26:40.832605 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.832583    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/858cf195-514b-452e-9d6a-57605def0790-web-config" (OuterVolumeSpecName: "web-config") pod "858cf195-514b-452e-9d6a-57605def0790" (UID: "858cf195-514b-452e-9d6a-57605def0790"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 22:26:40.905560 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.905529    2563 generic.go:356] "Generic (PLEG): container finished" podID="858cf195-514b-452e-9d6a-57605def0790" containerID="fe64a34ff1ef93e79f33f8efce518d20c05eff9edd2c3b0a85736d9d64c519ed" exitCode=0
Jul 06 22:26:40.905560 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.905560    2563 generic.go:356] "Generic (PLEG): container finished" podID="858cf195-514b-452e-9d6a-57605def0790" containerID="50d6658df0f9380712776d280a0f734937e525cc0d271a31088516bcc8f87798" exitCode=0
Jul 06 22:26:40.905736 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.905570    2563 generic.go:356] "Generic (PLEG): container finished" podID="858cf195-514b-452e-9d6a-57605def0790" containerID="3c4da65ff69af3667d1afb76aad721ec956224e9d1e8e53929a7852d5b3a24a0" exitCode=0
Jul 06 22:26:40.905736 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.905578    2563 generic.go:356] "Generic (PLEG): container finished" podID="858cf195-514b-452e-9d6a-57605def0790" containerID="423fdbf76002e11fda332a3bcf34104b6ee69f100484d4a09ed7de3b7cfa292a" exitCode=0
Jul 06 22:26:40.905736 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.905586    2563 generic.go:356] "Generic (PLEG): container finished" podID="858cf195-514b-452e-9d6a-57605def0790" containerID="caed834d4d1a297ab56e13de545d129aaaea1b58d1af37e019949ee91705ccf3" exitCode=0
Jul 06 22:26:40.905736 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.905594    2563 generic.go:356] "Generic (PLEG): container finished" podID="858cf195-514b-452e-9d6a-57605def0790" containerID="acf16978c8aac2bd1a48745d6d061e11bd3ff66a5766924fd30a3fa0f7c2161c" exitCode=0
Jul 06 22:26:40.905736 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.905615    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"858cf195-514b-452e-9d6a-57605def0790","Type":"ContainerDied","Data":"fe64a34ff1ef93e79f33f8efce518d20c05eff9edd2c3b0a85736d9d64c519ed"}
Jul 06 22:26:40.905736 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.905686    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"858cf195-514b-452e-9d6a-57605def0790","Type":"ContainerDied","Data":"50d6658df0f9380712776d280a0f734937e525cc0d271a31088516bcc8f87798"}
Jul 06 22:26:40.905736 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.905624    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 22:26:40.905736 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.905714    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"858cf195-514b-452e-9d6a-57605def0790","Type":"ContainerDied","Data":"3c4da65ff69af3667d1afb76aad721ec956224e9d1e8e53929a7852d5b3a24a0"}
Jul 06 22:26:40.905736 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.905739    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"858cf195-514b-452e-9d6a-57605def0790","Type":"ContainerDied","Data":"423fdbf76002e11fda332a3bcf34104b6ee69f100484d4a09ed7de3b7cfa292a"}
Jul 06 22:26:40.906014 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.905764    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"858cf195-514b-452e-9d6a-57605def0790","Type":"ContainerDied","Data":"caed834d4d1a297ab56e13de545d129aaaea1b58d1af37e019949ee91705ccf3"}
Jul 06 22:26:40.906014 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.905784    2563 scope.go:117] "RemoveContainer" containerID="fe64a34ff1ef93e79f33f8efce518d20c05eff9edd2c3b0a85736d9d64c519ed"
Jul 06 22:26:40.906014 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.905787    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"858cf195-514b-452e-9d6a-57605def0790","Type":"ContainerDied","Data":"acf16978c8aac2bd1a48745d6d061e11bd3ff66a5766924fd30a3fa0f7c2161c"}
Jul 06 22:26:40.906014 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.905893    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"858cf195-514b-452e-9d6a-57605def0790","Type":"ContainerDied","Data":"c218f60f9921e51c82d6a023ce3d931d9e62e7690a0185e809e1a5f19f008018"}
Jul 06 22:26:40.916536 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.916505    2563 scope.go:117] "RemoveContainer" containerID="50d6658df0f9380712776d280a0f734937e525cc0d271a31088516bcc8f87798"
Jul 06 22:26:40.920297 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.920275    2563 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/858cf195-514b-452e-9d6a-57605def0790-secret-metrics-client-certs\") on node \"ip-10-0-133-105.ec2.internal\" DevicePath \"\""
Jul 06 22:26:40.920381 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.920304    2563 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/858cf195-514b-452e-9d6a-57605def0790-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-133-105.ec2.internal\" DevicePath \"\""
Jul 06 22:26:40.920381 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.920321    2563 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/858cf195-514b-452e-9d6a-57605def0790-prometheus-k8s-db\") on node \"ip-10-0-133-105.ec2.internal\" DevicePath \"\""
Jul 06 22:26:40.920381 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.920334    2563 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/858cf195-514b-452e-9d6a-57605def0790-config-out\") on node \"ip-10-0-133-105.ec2.internal\" DevicePath \"\""
Jul 06 22:26:40.920381 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.920349    2563 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/858cf195-514b-452e-9d6a-57605def0790-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-133-105.ec2.internal\" DevicePath \"\""
Jul 06 22:26:40.920381 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.920364    2563 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/858cf195-514b-452e-9d6a-57605def0790-tls-assets\") on node \"ip-10-0-133-105.ec2.internal\" DevicePath \"\""
Jul 06 22:26:40.920381 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.920379    2563 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/858cf195-514b-452e-9d6a-57605def0790-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-133-105.ec2.internal\" DevicePath \"\""
Jul 06 22:26:40.920600 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.920394    2563 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/858cf195-514b-452e-9d6a-57605def0790-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-133-105.ec2.internal\" DevicePath \"\""
Jul 06 22:26:40.920600 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.920410    2563 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/858cf195-514b-452e-9d6a-57605def0790-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-133-105.ec2.internal\" DevicePath \"\""
Jul 06 22:26:40.920600 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.920424    2563 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/858cf195-514b-452e-9d6a-57605def0790-configmap-metrics-client-ca\") on node \"ip-10-0-133-105.ec2.internal\" DevicePath \"\""
Jul 06 22:26:40.920600 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.920436    2563 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/858cf195-514b-452e-9d6a-57605def0790-config\") on node \"ip-10-0-133-105.ec2.internal\" DevicePath \"\""
Jul 06 22:26:40.920600 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.920448    2563 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/858cf195-514b-452e-9d6a-57605def0790-web-config\") on node \"ip-10-0-133-105.ec2.internal\" DevicePath \"\""
Jul 06 22:26:40.920600 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.920463    2563 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/858cf195-514b-452e-9d6a-57605def0790-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-133-105.ec2.internal\" DevicePath \"\""
Jul 06 22:26:40.920600 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.920476    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-74z9p\" (UniqueName: \"kubernetes.io/projected/858cf195-514b-452e-9d6a-57605def0790-kube-api-access-74z9p\") on node \"ip-10-0-133-105.ec2.internal\" DevicePath \"\""
Jul 06 22:26:40.920600 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.920488    2563 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/858cf195-514b-452e-9d6a-57605def0790-thanos-prometheus-http-client-file\") on node \"ip-10-0-133-105.ec2.internal\" DevicePath \"\""
Jul 06 22:26:40.920600 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.920503    2563 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/858cf195-514b-452e-9d6a-57605def0790-secret-prometheus-k8s-tls\") on node \"ip-10-0-133-105.ec2.internal\" DevicePath \"\""
Jul 06 22:26:40.920600 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.920516    2563 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/858cf195-514b-452e-9d6a-57605def0790-secret-kube-rbac-proxy\") on node \"ip-10-0-133-105.ec2.internal\" DevicePath \"\""
Jul 06 22:26:40.920600 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.920529    2563 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/858cf195-514b-452e-9d6a-57605def0790-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-133-105.ec2.internal\" DevicePath \"\""
Jul 06 22:26:40.920600 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.920542    2563 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/858cf195-514b-452e-9d6a-57605def0790-prometheus-trusted-ca-bundle\") on node \"ip-10-0-133-105.ec2.internal\" DevicePath \"\""
Jul 06 22:26:40.920600 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.920555    2563 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/858cf195-514b-452e-9d6a-57605def0790-secret-grpc-tls\") on node \"ip-10-0-133-105.ec2.internal\" DevicePath \"\""
Jul 06 22:26:40.923847 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.923832    2563 scope.go:117] "RemoveContainer" containerID="3c4da65ff69af3667d1afb76aad721ec956224e9d1e8e53929a7852d5b3a24a0"
Jul 06 22:26:40.930129 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.930110    2563 scope.go:117] "RemoveContainer" containerID="423fdbf76002e11fda332a3bcf34104b6ee69f100484d4a09ed7de3b7cfa292a"
Jul 06 22:26:40.932422 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.932401    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 06 22:26:40.936815 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.936796    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 06 22:26:40.938030 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.938016    2563 scope.go:117] "RemoveContainer" containerID="caed834d4d1a297ab56e13de545d129aaaea1b58d1af37e019949ee91705ccf3"
Jul 06 22:26:40.944652 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.944625    2563 scope.go:117] "RemoveContainer" containerID="acf16978c8aac2bd1a48745d6d061e11bd3ff66a5766924fd30a3fa0f7c2161c"
Jul 06 22:26:40.951326 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.951308    2563 scope.go:117] "RemoveContainer" containerID="8dc4771a516d0840c7d49ae6c70d1994ffd6b8cb38ccbcbc2aa4a6e66cc39228"
Jul 06 22:26:40.961123 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.961100    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 06 22:26:40.961363 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.961350    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="858cf195-514b-452e-9d6a-57605def0790" containerName="thanos-sidecar"
Jul 06 22:26:40.961403 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.961364    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="858cf195-514b-452e-9d6a-57605def0790" containerName="thanos-sidecar"
Jul 06 22:26:40.961403 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.961373    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="858cf195-514b-452e-9d6a-57605def0790" containerName="config-reloader"
Jul 06 22:26:40.961403 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.961378    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="858cf195-514b-452e-9d6a-57605def0790" containerName="config-reloader"
Jul 06 22:26:40.961403 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.961387    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="858cf195-514b-452e-9d6a-57605def0790" containerName="kube-rbac-proxy-web"
Jul 06 22:26:40.961403 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.961393    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="858cf195-514b-452e-9d6a-57605def0790" containerName="kube-rbac-proxy-web"
Jul 06 22:26:40.961403 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.961402    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="858cf195-514b-452e-9d6a-57605def0790" containerName="kube-rbac-proxy"
Jul 06 22:26:40.961580 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.961407    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="858cf195-514b-452e-9d6a-57605def0790" containerName="kube-rbac-proxy"
Jul 06 22:26:40.961580 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.961421    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d019cfb8-f568-4a0e-be12-45f7557ac2c6" containerName="console"
Jul 06 22:26:40.961580 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.961426    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="d019cfb8-f568-4a0e-be12-45f7557ac2c6" containerName="console"
Jul 06 22:26:40.961580 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.961431    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="858cf195-514b-452e-9d6a-57605def0790" containerName="prometheus"
Jul 06 22:26:40.961580 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.961436    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="858cf195-514b-452e-9d6a-57605def0790" containerName="prometheus"
Jul 06 22:26:40.961580 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.961442    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="858cf195-514b-452e-9d6a-57605def0790" containerName="init-config-reloader"
Jul 06 22:26:40.961580 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.961448    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="858cf195-514b-452e-9d6a-57605def0790" containerName="init-config-reloader"
Jul 06 22:26:40.961580 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.961454    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="858cf195-514b-452e-9d6a-57605def0790" containerName="kube-rbac-proxy-thanos"
Jul 06 22:26:40.961580 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.961458    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="858cf195-514b-452e-9d6a-57605def0790" containerName="kube-rbac-proxy-thanos"
Jul 06 22:26:40.961580 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.961499    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="858cf195-514b-452e-9d6a-57605def0790" containerName="thanos-sidecar"
Jul 06 22:26:40.961580 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.961506    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="858cf195-514b-452e-9d6a-57605def0790" containerName="kube-rbac-proxy-web"
Jul 06 22:26:40.961580 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.961512    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="d019cfb8-f568-4a0e-be12-45f7557ac2c6" containerName="console"
Jul 06 22:26:40.961580 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.961518    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="858cf195-514b-452e-9d6a-57605def0790" containerName="config-reloader"
Jul 06 22:26:40.961580 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.961524    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="858cf195-514b-452e-9d6a-57605def0790" containerName="kube-rbac-proxy"
Jul 06 22:26:40.961580 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.961530    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="858cf195-514b-452e-9d6a-57605def0790" containerName="prometheus"
Jul 06 22:26:40.961580 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.961534    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="858cf195-514b-452e-9d6a-57605def0790" containerName="kube-rbac-proxy-thanos"
Jul 06 22:26:40.963061 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.962285    2563 scope.go:117] "RemoveContainer" containerID="fe64a34ff1ef93e79f33f8efce518d20c05eff9edd2c3b0a85736d9d64c519ed"
Jul 06 22:26:40.963543 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:26:40.963515    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fe64a34ff1ef93e79f33f8efce518d20c05eff9edd2c3b0a85736d9d64c519ed\": container with ID starting with fe64a34ff1ef93e79f33f8efce518d20c05eff9edd2c3b0a85736d9d64c519ed not found: ID does not exist" containerID="fe64a34ff1ef93e79f33f8efce518d20c05eff9edd2c3b0a85736d9d64c519ed"
Jul 06 22:26:40.963645 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.963553    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fe64a34ff1ef93e79f33f8efce518d20c05eff9edd2c3b0a85736d9d64c519ed"} err="failed to get container status \"fe64a34ff1ef93e79f33f8efce518d20c05eff9edd2c3b0a85736d9d64c519ed\": rpc error: code = NotFound desc = could not find container \"fe64a34ff1ef93e79f33f8efce518d20c05eff9edd2c3b0a85736d9d64c519ed\": container with ID starting with fe64a34ff1ef93e79f33f8efce518d20c05eff9edd2c3b0a85736d9d64c519ed not found: ID does not exist"
Jul 06 22:26:40.963645 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.963585    2563 scope.go:117] "RemoveContainer" containerID="50d6658df0f9380712776d280a0f734937e525cc0d271a31088516bcc8f87798"
Jul 06 22:26:40.967988 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:26:40.966087    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"50d6658df0f9380712776d280a0f734937e525cc0d271a31088516bcc8f87798\": container with ID starting with 50d6658df0f9380712776d280a0f734937e525cc0d271a31088516bcc8f87798 not found: ID does not exist" containerID="50d6658df0f9380712776d280a0f734937e525cc0d271a31088516bcc8f87798"
Jul 06 22:26:40.967988 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.966130    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"50d6658df0f9380712776d280a0f734937e525cc0d271a31088516bcc8f87798"} err="failed to get container status \"50d6658df0f9380712776d280a0f734937e525cc0d271a31088516bcc8f87798\": rpc error: code = NotFound desc = could not find container \"50d6658df0f9380712776d280a0f734937e525cc0d271a31088516bcc8f87798\": container with ID starting with 50d6658df0f9380712776d280a0f734937e525cc0d271a31088516bcc8f87798 not found: ID does not exist"
Jul 06 22:26:40.967988 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.966154    2563 scope.go:117] "RemoveContainer" containerID="3c4da65ff69af3667d1afb76aad721ec956224e9d1e8e53929a7852d5b3a24a0"
Jul 06 22:26:40.967988 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:26:40.966485    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3c4da65ff69af3667d1afb76aad721ec956224e9d1e8e53929a7852d5b3a24a0\": container with ID starting with 3c4da65ff69af3667d1afb76aad721ec956224e9d1e8e53929a7852d5b3a24a0 not found: ID does not exist" containerID="3c4da65ff69af3667d1afb76aad721ec956224e9d1e8e53929a7852d5b3a24a0"
Jul 06 22:26:40.967988 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.966524    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3c4da65ff69af3667d1afb76aad721ec956224e9d1e8e53929a7852d5b3a24a0"} err="failed to get container status \"3c4da65ff69af3667d1afb76aad721ec956224e9d1e8e53929a7852d5b3a24a0\": rpc error: code = NotFound desc = could not find container \"3c4da65ff69af3667d1afb76aad721ec956224e9d1e8e53929a7852d5b3a24a0\": container with ID starting with 3c4da65ff69af3667d1afb76aad721ec956224e9d1e8e53929a7852d5b3a24a0 not found: ID does not exist"
Jul 06 22:26:40.967988 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.966544    2563 scope.go:117] "RemoveContainer" containerID="423fdbf76002e11fda332a3bcf34104b6ee69f100484d4a09ed7de3b7cfa292a"
Jul 06 22:26:40.967988 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.966810    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 22:26:40.967988 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:26:40.966863    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"423fdbf76002e11fda332a3bcf34104b6ee69f100484d4a09ed7de3b7cfa292a\": container with ID starting with 423fdbf76002e11fda332a3bcf34104b6ee69f100484d4a09ed7de3b7cfa292a not found: ID does not exist" containerID="423fdbf76002e11fda332a3bcf34104b6ee69f100484d4a09ed7de3b7cfa292a"
Jul 06 22:26:40.967988 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.966897    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"423fdbf76002e11fda332a3bcf34104b6ee69f100484d4a09ed7de3b7cfa292a"} err="failed to get container status \"423fdbf76002e11fda332a3bcf34104b6ee69f100484d4a09ed7de3b7cfa292a\": rpc error: code = NotFound desc = could not find container \"423fdbf76002e11fda332a3bcf34104b6ee69f100484d4a09ed7de3b7cfa292a\": container with ID starting with 423fdbf76002e11fda332a3bcf34104b6ee69f100484d4a09ed7de3b7cfa292a not found: ID does not exist"
Jul 06 22:26:40.967988 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.966913    2563 scope.go:117] "RemoveContainer" containerID="caed834d4d1a297ab56e13de545d129aaaea1b58d1af37e019949ee91705ccf3"
Jul 06 22:26:40.967988 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:26:40.967288    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"caed834d4d1a297ab56e13de545d129aaaea1b58d1af37e019949ee91705ccf3\": container with ID starting with caed834d4d1a297ab56e13de545d129aaaea1b58d1af37e019949ee91705ccf3 not found: ID does not exist" containerID="caed834d4d1a297ab56e13de545d129aaaea1b58d1af37e019949ee91705ccf3"
Jul 06 22:26:40.967988 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.967319    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"caed834d4d1a297ab56e13de545d129aaaea1b58d1af37e019949ee91705ccf3"} err="failed to get container status \"caed834d4d1a297ab56e13de545d129aaaea1b58d1af37e019949ee91705ccf3\": rpc error: code = NotFound desc = could not find container \"caed834d4d1a297ab56e13de545d129aaaea1b58d1af37e019949ee91705ccf3\": container with ID starting with caed834d4d1a297ab56e13de545d129aaaea1b58d1af37e019949ee91705ccf3 not found: ID does not exist"
Jul 06 22:26:40.967988 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.967349    2563 scope.go:117] "RemoveContainer" containerID="acf16978c8aac2bd1a48745d6d061e11bd3ff66a5766924fd30a3fa0f7c2161c"
Jul 06 22:26:40.967988 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:26:40.967759    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"acf16978c8aac2bd1a48745d6d061e11bd3ff66a5766924fd30a3fa0f7c2161c\": container with ID starting with acf16978c8aac2bd1a48745d6d061e11bd3ff66a5766924fd30a3fa0f7c2161c not found: ID does not exist" containerID="acf16978c8aac2bd1a48745d6d061e11bd3ff66a5766924fd30a3fa0f7c2161c"
Jul 06 22:26:40.967988 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.967800    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"acf16978c8aac2bd1a48745d6d061e11bd3ff66a5766924fd30a3fa0f7c2161c"} err="failed to get container status \"acf16978c8aac2bd1a48745d6d061e11bd3ff66a5766924fd30a3fa0f7c2161c\": rpc error: code = NotFound desc = could not find container \"acf16978c8aac2bd1a48745d6d061e11bd3ff66a5766924fd30a3fa0f7c2161c\": container with ID starting with acf16978c8aac2bd1a48745d6d061e11bd3ff66a5766924fd30a3fa0f7c2161c not found: ID does not exist"
Jul 06 22:26:40.967988 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.967822    2563 scope.go:117] "RemoveContainer" containerID="8dc4771a516d0840c7d49ae6c70d1994ffd6b8cb38ccbcbc2aa4a6e66cc39228"
Jul 06 22:26:40.968745 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:26:40.968565    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8dc4771a516d0840c7d49ae6c70d1994ffd6b8cb38ccbcbc2aa4a6e66cc39228\": container with ID starting with 8dc4771a516d0840c7d49ae6c70d1994ffd6b8cb38ccbcbc2aa4a6e66cc39228 not found: ID does not exist" containerID="8dc4771a516d0840c7d49ae6c70d1994ffd6b8cb38ccbcbc2aa4a6e66cc39228"
Jul 06 22:26:40.968745 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.968669    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8dc4771a516d0840c7d49ae6c70d1994ffd6b8cb38ccbcbc2aa4a6e66cc39228"} err="failed to get container status \"8dc4771a516d0840c7d49ae6c70d1994ffd6b8cb38ccbcbc2aa4a6e66cc39228\": rpc error: code = NotFound desc = could not find container \"8dc4771a516d0840c7d49ae6c70d1994ffd6b8cb38ccbcbc2aa4a6e66cc39228\": container with ID starting with 8dc4771a516d0840c7d49ae6c70d1994ffd6b8cb38ccbcbc2aa4a6e66cc39228 not found: ID does not exist"
Jul 06 22:26:40.968887 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.968699    2563 scope.go:117] "RemoveContainer" containerID="fe64a34ff1ef93e79f33f8efce518d20c05eff9edd2c3b0a85736d9d64c519ed"
Jul 06 22:26:40.969343 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.969299    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fe64a34ff1ef93e79f33f8efce518d20c05eff9edd2c3b0a85736d9d64c519ed"} err="failed to get container status \"fe64a34ff1ef93e79f33f8efce518d20c05eff9edd2c3b0a85736d9d64c519ed\": rpc error: code = NotFound desc = could not find container \"fe64a34ff1ef93e79f33f8efce518d20c05eff9edd2c3b0a85736d9d64c519ed\": container with ID starting with fe64a34ff1ef93e79f33f8efce518d20c05eff9edd2c3b0a85736d9d64c519ed not found: ID does not exist"
Jul 06 22:26:40.969532 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.969346    2563 scope.go:117] "RemoveContainer" containerID="50d6658df0f9380712776d280a0f734937e525cc0d271a31088516bcc8f87798"
Jul 06 22:26:40.969532 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.969429    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 06 22:26:40.969710 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.969690    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 06 22:26:40.969789 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.969733    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-3hmca37nvuqpp\""
Jul 06 22:26:40.969899 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.969880    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 06 22:26:40.970004 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.969966    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"50d6658df0f9380712776d280a0f734937e525cc0d271a31088516bcc8f87798"} err="failed to get container status \"50d6658df0f9380712776d280a0f734937e525cc0d271a31088516bcc8f87798\": rpc error: code = NotFound desc = could not find container \"50d6658df0f9380712776d280a0f734937e525cc0d271a31088516bcc8f87798\": container with ID starting with 50d6658df0f9380712776d280a0f734937e525cc0d271a31088516bcc8f87798 not found: ID does not exist"
Jul 06 22:26:40.970087 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.970028    2563 scope.go:117] "RemoveContainer" containerID="3c4da65ff69af3667d1afb76aad721ec956224e9d1e8e53929a7852d5b3a24a0"
Jul 06 22:26:40.970375 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.970350    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3c4da65ff69af3667d1afb76aad721ec956224e9d1e8e53929a7852d5b3a24a0"} err="failed to get container status \"3c4da65ff69af3667d1afb76aad721ec956224e9d1e8e53929a7852d5b3a24a0\": rpc error: code = NotFound desc = could not find container \"3c4da65ff69af3667d1afb76aad721ec956224e9d1e8e53929a7852d5b3a24a0\": container with ID starting with 3c4da65ff69af3667d1afb76aad721ec956224e9d1e8e53929a7852d5b3a24a0 not found: ID does not exist"
Jul 06 22:26:40.970472 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.970374    2563 scope.go:117] "RemoveContainer" containerID="423fdbf76002e11fda332a3bcf34104b6ee69f100484d4a09ed7de3b7cfa292a"
Jul 06 22:26:40.970472 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.970424    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 06 22:26:40.970626 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.970529    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 06 22:26:40.970709 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.970670    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 06 22:26:40.970806 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.970722    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"423fdbf76002e11fda332a3bcf34104b6ee69f100484d4a09ed7de3b7cfa292a"} err="failed to get container status \"423fdbf76002e11fda332a3bcf34104b6ee69f100484d4a09ed7de3b7cfa292a\": rpc error: code = NotFound desc = could not find container \"423fdbf76002e11fda332a3bcf34104b6ee69f100484d4a09ed7de3b7cfa292a\": container with ID starting with 423fdbf76002e11fda332a3bcf34104b6ee69f100484d4a09ed7de3b7cfa292a not found: ID does not exist"
Jul 06 22:26:40.970806 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.970753    2563 scope.go:117] "RemoveContainer" containerID="caed834d4d1a297ab56e13de545d129aaaea1b58d1af37e019949ee91705ccf3"
Jul 06 22:26:40.970806 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.970790    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 06 22:26:40.971182 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.971123    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 06 22:26:40.971242 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.971156    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 06 22:26:40.971445 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.971414    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"caed834d4d1a297ab56e13de545d129aaaea1b58d1af37e019949ee91705ccf3"} err="failed to get container status \"caed834d4d1a297ab56e13de545d129aaaea1b58d1af37e019949ee91705ccf3\": rpc error: code = NotFound desc = could not find container \"caed834d4d1a297ab56e13de545d129aaaea1b58d1af37e019949ee91705ccf3\": container with ID starting with caed834d4d1a297ab56e13de545d129aaaea1b58d1af37e019949ee91705ccf3 not found: ID does not exist"
Jul 06 22:26:40.971445 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.971438    2563 scope.go:117] "RemoveContainer" containerID="acf16978c8aac2bd1a48745d6d061e11bd3ff66a5766924fd30a3fa0f7c2161c"
Jul 06 22:26:40.971773 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.971746    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"acf16978c8aac2bd1a48745d6d061e11bd3ff66a5766924fd30a3fa0f7c2161c"} err="failed to get container status \"acf16978c8aac2bd1a48745d6d061e11bd3ff66a5766924fd30a3fa0f7c2161c\": rpc error: code = NotFound desc = could not find container \"acf16978c8aac2bd1a48745d6d061e11bd3ff66a5766924fd30a3fa0f7c2161c\": container with ID starting with acf16978c8aac2bd1a48745d6d061e11bd3ff66a5766924fd30a3fa0f7c2161c not found: ID does not exist"
Jul 06 22:26:40.971892 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.971769    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 06 22:26:40.971892 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.971778    2563 scope.go:117] "RemoveContainer" containerID="8dc4771a516d0840c7d49ae6c70d1994ffd6b8cb38ccbcbc2aa4a6e66cc39228"
Jul 06 22:26:40.972008 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.971896    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 06 22:26:40.972008 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.971906    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 06 22:26:40.972008 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.971964    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 06 22:26:40.972259 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.972009    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-6shnk\""
Jul 06 22:26:40.972364 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.972291    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8dc4771a516d0840c7d49ae6c70d1994ffd6b8cb38ccbcbc2aa4a6e66cc39228"} err="failed to get container status \"8dc4771a516d0840c7d49ae6c70d1994ffd6b8cb38ccbcbc2aa4a6e66cc39228\": rpc error: code = NotFound desc = could not find container \"8dc4771a516d0840c7d49ae6c70d1994ffd6b8cb38ccbcbc2aa4a6e66cc39228\": container with ID starting with 8dc4771a516d0840c7d49ae6c70d1994ffd6b8cb38ccbcbc2aa4a6e66cc39228 not found: ID does not exist"
Jul 06 22:26:40.972364 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.972307    2563 scope.go:117] "RemoveContainer" containerID="fe64a34ff1ef93e79f33f8efce518d20c05eff9edd2c3b0a85736d9d64c519ed"
Jul 06 22:26:40.972581 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.972562    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fe64a34ff1ef93e79f33f8efce518d20c05eff9edd2c3b0a85736d9d64c519ed"} err="failed to get container status \"fe64a34ff1ef93e79f33f8efce518d20c05eff9edd2c3b0a85736d9d64c519ed\": rpc error: code = NotFound desc = could not find container \"fe64a34ff1ef93e79f33f8efce518d20c05eff9edd2c3b0a85736d9d64c519ed\": container with ID starting with fe64a34ff1ef93e79f33f8efce518d20c05eff9edd2c3b0a85736d9d64c519ed not found: ID does not exist"
Jul 06 22:26:40.972701 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.972584    2563 scope.go:117] "RemoveContainer" containerID="50d6658df0f9380712776d280a0f734937e525cc0d271a31088516bcc8f87798"
Jul 06 22:26:40.972877 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.972844    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"50d6658df0f9380712776d280a0f734937e525cc0d271a31088516bcc8f87798"} err="failed to get container status \"50d6658df0f9380712776d280a0f734937e525cc0d271a31088516bcc8f87798\": rpc error: code = NotFound desc = could not find container \"50d6658df0f9380712776d280a0f734937e525cc0d271a31088516bcc8f87798\": container with ID starting with 50d6658df0f9380712776d280a0f734937e525cc0d271a31088516bcc8f87798 not found: ID does not exist"
Jul 06 22:26:40.972954 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.972885    2563 scope.go:117] "RemoveContainer" containerID="3c4da65ff69af3667d1afb76aad721ec956224e9d1e8e53929a7852d5b3a24a0"
Jul 06 22:26:40.973223 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.973141    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3c4da65ff69af3667d1afb76aad721ec956224e9d1e8e53929a7852d5b3a24a0"} err="failed to get container status \"3c4da65ff69af3667d1afb76aad721ec956224e9d1e8e53929a7852d5b3a24a0\": rpc error: code = NotFound desc = could not find container \"3c4da65ff69af3667d1afb76aad721ec956224e9d1e8e53929a7852d5b3a24a0\": container with ID starting with 3c4da65ff69af3667d1afb76aad721ec956224e9d1e8e53929a7852d5b3a24a0 not found: ID does not exist"
Jul 06 22:26:40.973223 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.973194    2563 scope.go:117] "RemoveContainer" containerID="423fdbf76002e11fda332a3bcf34104b6ee69f100484d4a09ed7de3b7cfa292a"
Jul 06 22:26:40.973472 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.973435    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"423fdbf76002e11fda332a3bcf34104b6ee69f100484d4a09ed7de3b7cfa292a"} err="failed to get container status \"423fdbf76002e11fda332a3bcf34104b6ee69f100484d4a09ed7de3b7cfa292a\": rpc error: code = NotFound desc = could not find container \"423fdbf76002e11fda332a3bcf34104b6ee69f100484d4a09ed7de3b7cfa292a\": container with ID starting with 423fdbf76002e11fda332a3bcf34104b6ee69f100484d4a09ed7de3b7cfa292a not found: ID does not exist"
Jul 06 22:26:40.973472 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.973461    2563 scope.go:117] "RemoveContainer" containerID="caed834d4d1a297ab56e13de545d129aaaea1b58d1af37e019949ee91705ccf3"
Jul 06 22:26:40.973722 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.973699    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"caed834d4d1a297ab56e13de545d129aaaea1b58d1af37e019949ee91705ccf3"} err="failed to get container status \"caed834d4d1a297ab56e13de545d129aaaea1b58d1af37e019949ee91705ccf3\": rpc error: code = NotFound desc = could not find container \"caed834d4d1a297ab56e13de545d129aaaea1b58d1af37e019949ee91705ccf3\": container with ID starting with caed834d4d1a297ab56e13de545d129aaaea1b58d1af37e019949ee91705ccf3 not found: ID does not exist"
Jul 06 22:26:40.973813 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.973726    2563 scope.go:117] "RemoveContainer" containerID="acf16978c8aac2bd1a48745d6d061e11bd3ff66a5766924fd30a3fa0f7c2161c"
Jul 06 22:26:40.973813 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.973739    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 06 22:26:40.974040 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.973988    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"acf16978c8aac2bd1a48745d6d061e11bd3ff66a5766924fd30a3fa0f7c2161c"} err="failed to get container status \"acf16978c8aac2bd1a48745d6d061e11bd3ff66a5766924fd30a3fa0f7c2161c\": rpc error: code = NotFound desc = could not find container \"acf16978c8aac2bd1a48745d6d061e11bd3ff66a5766924fd30a3fa0f7c2161c\": container with ID starting with acf16978c8aac2bd1a48745d6d061e11bd3ff66a5766924fd30a3fa0f7c2161c not found: ID does not exist"
Jul 06 22:26:40.974040 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.974012    2563 scope.go:117] "RemoveContainer" containerID="8dc4771a516d0840c7d49ae6c70d1994ffd6b8cb38ccbcbc2aa4a6e66cc39228"
Jul 06 22:26:40.974313 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.974289    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8dc4771a516d0840c7d49ae6c70d1994ffd6b8cb38ccbcbc2aa4a6e66cc39228"} err="failed to get container status \"8dc4771a516d0840c7d49ae6c70d1994ffd6b8cb38ccbcbc2aa4a6e66cc39228\": rpc error: code = NotFound desc = could not find container \"8dc4771a516d0840c7d49ae6c70d1994ffd6b8cb38ccbcbc2aa4a6e66cc39228\": container with ID starting with 8dc4771a516d0840c7d49ae6c70d1994ffd6b8cb38ccbcbc2aa4a6e66cc39228 not found: ID does not exist"
Jul 06 22:26:40.974441 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.974315    2563 scope.go:117] "RemoveContainer" containerID="fe64a34ff1ef93e79f33f8efce518d20c05eff9edd2c3b0a85736d9d64c519ed"
Jul 06 22:26:40.974642 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.974619    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fe64a34ff1ef93e79f33f8efce518d20c05eff9edd2c3b0a85736d9d64c519ed"} err="failed to get container status \"fe64a34ff1ef93e79f33f8efce518d20c05eff9edd2c3b0a85736d9d64c519ed\": rpc error: code = NotFound desc = could not find container \"fe64a34ff1ef93e79f33f8efce518d20c05eff9edd2c3b0a85736d9d64c519ed\": container with ID starting with fe64a34ff1ef93e79f33f8efce518d20c05eff9edd2c3b0a85736d9d64c519ed not found: ID does not exist"
Jul 06 22:26:40.974769 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.974645    2563 scope.go:117] "RemoveContainer" containerID="50d6658df0f9380712776d280a0f734937e525cc0d271a31088516bcc8f87798"
Jul 06 22:26:40.974930 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.974908    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"50d6658df0f9380712776d280a0f734937e525cc0d271a31088516bcc8f87798"} err="failed to get container status \"50d6658df0f9380712776d280a0f734937e525cc0d271a31088516bcc8f87798\": rpc error: code = NotFound desc = could not find container \"50d6658df0f9380712776d280a0f734937e525cc0d271a31088516bcc8f87798\": container with ID starting with 50d6658df0f9380712776d280a0f734937e525cc0d271a31088516bcc8f87798 not found: ID does not exist"
Jul 06 22:26:40.974930 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.974928    2563 scope.go:117] "RemoveContainer" containerID="3c4da65ff69af3667d1afb76aad721ec956224e9d1e8e53929a7852d5b3a24a0"
Jul 06 22:26:40.975238 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.975220    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3c4da65ff69af3667d1afb76aad721ec956224e9d1e8e53929a7852d5b3a24a0"} err="failed to get container status \"3c4da65ff69af3667d1afb76aad721ec956224e9d1e8e53929a7852d5b3a24a0\": rpc error: code = NotFound desc = could not find container \"3c4da65ff69af3667d1afb76aad721ec956224e9d1e8e53929a7852d5b3a24a0\": container with ID starting with 3c4da65ff69af3667d1afb76aad721ec956224e9d1e8e53929a7852d5b3a24a0 not found: ID does not exist"
Jul 06 22:26:40.975238 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.975238    2563 scope.go:117] "RemoveContainer" containerID="423fdbf76002e11fda332a3bcf34104b6ee69f100484d4a09ed7de3b7cfa292a"
Jul 06 22:26:40.975563 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.975537    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"423fdbf76002e11fda332a3bcf34104b6ee69f100484d4a09ed7de3b7cfa292a"} err="failed to get container status \"423fdbf76002e11fda332a3bcf34104b6ee69f100484d4a09ed7de3b7cfa292a\": rpc error: code = NotFound desc = could not find container \"423fdbf76002e11fda332a3bcf34104b6ee69f100484d4a09ed7de3b7cfa292a\": container with ID starting with 423fdbf76002e11fda332a3bcf34104b6ee69f100484d4a09ed7de3b7cfa292a not found: ID does not exist"
Jul 06 22:26:40.975630 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.975571    2563 scope.go:117] "RemoveContainer" containerID="caed834d4d1a297ab56e13de545d129aaaea1b58d1af37e019949ee91705ccf3"
Jul 06 22:26:40.975937 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.975913    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"caed834d4d1a297ab56e13de545d129aaaea1b58d1af37e019949ee91705ccf3"} err="failed to get container status \"caed834d4d1a297ab56e13de545d129aaaea1b58d1af37e019949ee91705ccf3\": rpc error: code = NotFound desc = could not find container \"caed834d4d1a297ab56e13de545d129aaaea1b58d1af37e019949ee91705ccf3\": container with ID starting with caed834d4d1a297ab56e13de545d129aaaea1b58d1af37e019949ee91705ccf3 not found: ID does not exist"
Jul 06 22:26:40.976023 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.975938    2563 scope.go:117] "RemoveContainer" containerID="acf16978c8aac2bd1a48745d6d061e11bd3ff66a5766924fd30a3fa0f7c2161c"
Jul 06 22:26:40.976224 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.976202    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"acf16978c8aac2bd1a48745d6d061e11bd3ff66a5766924fd30a3fa0f7c2161c"} err="failed to get container status \"acf16978c8aac2bd1a48745d6d061e11bd3ff66a5766924fd30a3fa0f7c2161c\": rpc error: code = NotFound desc = could not find container \"acf16978c8aac2bd1a48745d6d061e11bd3ff66a5766924fd30a3fa0f7c2161c\": container with ID starting with acf16978c8aac2bd1a48745d6d061e11bd3ff66a5766924fd30a3fa0f7c2161c not found: ID does not exist"
Jul 06 22:26:40.976288 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.976232    2563 scope.go:117] "RemoveContainer" containerID="8dc4771a516d0840c7d49ae6c70d1994ffd6b8cb38ccbcbc2aa4a6e66cc39228"
Jul 06 22:26:40.976489 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.976464    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8dc4771a516d0840c7d49ae6c70d1994ffd6b8cb38ccbcbc2aa4a6e66cc39228"} err="failed to get container status \"8dc4771a516d0840c7d49ae6c70d1994ffd6b8cb38ccbcbc2aa4a6e66cc39228\": rpc error: code = NotFound desc = could not find container \"8dc4771a516d0840c7d49ae6c70d1994ffd6b8cb38ccbcbc2aa4a6e66cc39228\": container with ID starting with 8dc4771a516d0840c7d49ae6c70d1994ffd6b8cb38ccbcbc2aa4a6e66cc39228 not found: ID does not exist"
Jul 06 22:26:40.976554 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.976491    2563 scope.go:117] "RemoveContainer" containerID="fe64a34ff1ef93e79f33f8efce518d20c05eff9edd2c3b0a85736d9d64c519ed"
Jul 06 22:26:40.976759 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.976741    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fe64a34ff1ef93e79f33f8efce518d20c05eff9edd2c3b0a85736d9d64c519ed"} err="failed to get container status \"fe64a34ff1ef93e79f33f8efce518d20c05eff9edd2c3b0a85736d9d64c519ed\": rpc error: code = NotFound desc = could not find container \"fe64a34ff1ef93e79f33f8efce518d20c05eff9edd2c3b0a85736d9d64c519ed\": container with ID starting with fe64a34ff1ef93e79f33f8efce518d20c05eff9edd2c3b0a85736d9d64c519ed not found: ID does not exist"
Jul 06 22:26:40.976815 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.976761    2563 scope.go:117] "RemoveContainer" containerID="50d6658df0f9380712776d280a0f734937e525cc0d271a31088516bcc8f87798"
Jul 06 22:26:40.977073 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.977049    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"50d6658df0f9380712776d280a0f734937e525cc0d271a31088516bcc8f87798"} err="failed to get container status \"50d6658df0f9380712776d280a0f734937e525cc0d271a31088516bcc8f87798\": rpc error: code = NotFound desc = could not find container \"50d6658df0f9380712776d280a0f734937e525cc0d271a31088516bcc8f87798\": container with ID starting with 50d6658df0f9380712776d280a0f734937e525cc0d271a31088516bcc8f87798 not found: ID does not exist"
Jul 06 22:26:40.977120 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.977076    2563 scope.go:117] "RemoveContainer" containerID="3c4da65ff69af3667d1afb76aad721ec956224e9d1e8e53929a7852d5b3a24a0"
Jul 06 22:26:40.977324 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.977304    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3c4da65ff69af3667d1afb76aad721ec956224e9d1e8e53929a7852d5b3a24a0"} err="failed to get container status \"3c4da65ff69af3667d1afb76aad721ec956224e9d1e8e53929a7852d5b3a24a0\": rpc error: code = NotFound desc = could not find container \"3c4da65ff69af3667d1afb76aad721ec956224e9d1e8e53929a7852d5b3a24a0\": container with ID starting with 3c4da65ff69af3667d1afb76aad721ec956224e9d1e8e53929a7852d5b3a24a0 not found: ID does not exist"
Jul 06 22:26:40.977374 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.977327    2563 scope.go:117] "RemoveContainer" containerID="423fdbf76002e11fda332a3bcf34104b6ee69f100484d4a09ed7de3b7cfa292a"
Jul 06 22:26:40.977566 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.977541    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"423fdbf76002e11fda332a3bcf34104b6ee69f100484d4a09ed7de3b7cfa292a"} err="failed to get container status \"423fdbf76002e11fda332a3bcf34104b6ee69f100484d4a09ed7de3b7cfa292a\": rpc error: code = NotFound desc = could not find container \"423fdbf76002e11fda332a3bcf34104b6ee69f100484d4a09ed7de3b7cfa292a\": container with ID starting with 423fdbf76002e11fda332a3bcf34104b6ee69f100484d4a09ed7de3b7cfa292a not found: ID does not exist"
Jul 06 22:26:40.977612 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.977569    2563 scope.go:117] "RemoveContainer" containerID="caed834d4d1a297ab56e13de545d129aaaea1b58d1af37e019949ee91705ccf3"
Jul 06 22:26:40.977680 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.977665    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 06 22:26:40.977832 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.977812    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"caed834d4d1a297ab56e13de545d129aaaea1b58d1af37e019949ee91705ccf3"} err="failed to get container status \"caed834d4d1a297ab56e13de545d129aaaea1b58d1af37e019949ee91705ccf3\": rpc error: code = NotFound desc = could not find container \"caed834d4d1a297ab56e13de545d129aaaea1b58d1af37e019949ee91705ccf3\": container with ID starting with caed834d4d1a297ab56e13de545d129aaaea1b58d1af37e019949ee91705ccf3 not found: ID does not exist"
Jul 06 22:26:40.977832 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.977832    2563 scope.go:117] "RemoveContainer" containerID="acf16978c8aac2bd1a48745d6d061e11bd3ff66a5766924fd30a3fa0f7c2161c"
Jul 06 22:26:40.978037 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.978015    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"acf16978c8aac2bd1a48745d6d061e11bd3ff66a5766924fd30a3fa0f7c2161c"} err="failed to get container status \"acf16978c8aac2bd1a48745d6d061e11bd3ff66a5766924fd30a3fa0f7c2161c\": rpc error: code = NotFound desc = could not find container \"acf16978c8aac2bd1a48745d6d061e11bd3ff66a5766924fd30a3fa0f7c2161c\": container with ID starting with acf16978c8aac2bd1a48745d6d061e11bd3ff66a5766924fd30a3fa0f7c2161c not found: ID does not exist"
Jul 06 22:26:40.978106 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.978039    2563 scope.go:117] "RemoveContainer" containerID="8dc4771a516d0840c7d49ae6c70d1994ffd6b8cb38ccbcbc2aa4a6e66cc39228"
Jul 06 22:26:40.978383 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.978362    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8dc4771a516d0840c7d49ae6c70d1994ffd6b8cb38ccbcbc2aa4a6e66cc39228"} err="failed to get container status \"8dc4771a516d0840c7d49ae6c70d1994ffd6b8cb38ccbcbc2aa4a6e66cc39228\": rpc error: code = NotFound desc = could not find container \"8dc4771a516d0840c7d49ae6c70d1994ffd6b8cb38ccbcbc2aa4a6e66cc39228\": container with ID starting with 8dc4771a516d0840c7d49ae6c70d1994ffd6b8cb38ccbcbc2aa4a6e66cc39228 not found: ID does not exist"
Jul 06 22:26:40.978463 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.978403    2563 scope.go:117] "RemoveContainer" containerID="fe64a34ff1ef93e79f33f8efce518d20c05eff9edd2c3b0a85736d9d64c519ed"
Jul 06 22:26:40.978781 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.978758    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fe64a34ff1ef93e79f33f8efce518d20c05eff9edd2c3b0a85736d9d64c519ed"} err="failed to get container status \"fe64a34ff1ef93e79f33f8efce518d20c05eff9edd2c3b0a85736d9d64c519ed\": rpc error: code = NotFound desc = could not find container \"fe64a34ff1ef93e79f33f8efce518d20c05eff9edd2c3b0a85736d9d64c519ed\": container with ID starting with fe64a34ff1ef93e79f33f8efce518d20c05eff9edd2c3b0a85736d9d64c519ed not found: ID does not exist"
Jul 06 22:26:40.979434 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.979411    2563 scope.go:117] "RemoveContainer" containerID="50d6658df0f9380712776d280a0f734937e525cc0d271a31088516bcc8f87798"
Jul 06 22:26:40.979687 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.979665    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"50d6658df0f9380712776d280a0f734937e525cc0d271a31088516bcc8f87798"} err="failed to get container status \"50d6658df0f9380712776d280a0f734937e525cc0d271a31088516bcc8f87798\": rpc error: code = NotFound desc = could not find container \"50d6658df0f9380712776d280a0f734937e525cc0d271a31088516bcc8f87798\": container with ID starting with 50d6658df0f9380712776d280a0f734937e525cc0d271a31088516bcc8f87798 not found: ID does not exist"
Jul 06 22:26:40.979755 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.979689    2563 scope.go:117] "RemoveContainer" containerID="3c4da65ff69af3667d1afb76aad721ec956224e9d1e8e53929a7852d5b3a24a0"
Jul 06 22:26:40.979928 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.979910    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3c4da65ff69af3667d1afb76aad721ec956224e9d1e8e53929a7852d5b3a24a0"} err="failed to get container status \"3c4da65ff69af3667d1afb76aad721ec956224e9d1e8e53929a7852d5b3a24a0\": rpc error: code = NotFound desc = could not find container \"3c4da65ff69af3667d1afb76aad721ec956224e9d1e8e53929a7852d5b3a24a0\": container with ID starting with 3c4da65ff69af3667d1afb76aad721ec956224e9d1e8e53929a7852d5b3a24a0 not found: ID does not exist"
Jul 06 22:26:40.979981 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.979929    2563 scope.go:117] "RemoveContainer" containerID="423fdbf76002e11fda332a3bcf34104b6ee69f100484d4a09ed7de3b7cfa292a"
Jul 06 22:26:40.980217 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.980197    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"423fdbf76002e11fda332a3bcf34104b6ee69f100484d4a09ed7de3b7cfa292a"} err="failed to get container status \"423fdbf76002e11fda332a3bcf34104b6ee69f100484d4a09ed7de3b7cfa292a\": rpc error: code = NotFound desc = could not find container \"423fdbf76002e11fda332a3bcf34104b6ee69f100484d4a09ed7de3b7cfa292a\": container with ID starting with 423fdbf76002e11fda332a3bcf34104b6ee69f100484d4a09ed7de3b7cfa292a not found: ID does not exist"
Jul 06 22:26:40.980301 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.980218    2563 scope.go:117] "RemoveContainer" containerID="caed834d4d1a297ab56e13de545d129aaaea1b58d1af37e019949ee91705ccf3"
Jul 06 22:26:40.980492 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.980469    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"caed834d4d1a297ab56e13de545d129aaaea1b58d1af37e019949ee91705ccf3"} err="failed to get container status \"caed834d4d1a297ab56e13de545d129aaaea1b58d1af37e019949ee91705ccf3\": rpc error: code = NotFound desc = could not find container \"caed834d4d1a297ab56e13de545d129aaaea1b58d1af37e019949ee91705ccf3\": container with ID starting with caed834d4d1a297ab56e13de545d129aaaea1b58d1af37e019949ee91705ccf3 not found: ID does not exist"
Jul 06 22:26:40.980492 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.980491    2563 scope.go:117] "RemoveContainer" containerID="acf16978c8aac2bd1a48745d6d061e11bd3ff66a5766924fd30a3fa0f7c2161c"
Jul 06 22:26:40.980766 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.980738    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"acf16978c8aac2bd1a48745d6d061e11bd3ff66a5766924fd30a3fa0f7c2161c"} err="failed to get container status \"acf16978c8aac2bd1a48745d6d061e11bd3ff66a5766924fd30a3fa0f7c2161c\": rpc error: code = NotFound desc = could not find container \"acf16978c8aac2bd1a48745d6d061e11bd3ff66a5766924fd30a3fa0f7c2161c\": container with ID starting with acf16978c8aac2bd1a48745d6d061e11bd3ff66a5766924fd30a3fa0f7c2161c not found: ID does not exist"
Jul 06 22:26:40.980820 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.980761    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 06 22:26:40.980855 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.980767    2563 scope.go:117] "RemoveContainer" containerID="8dc4771a516d0840c7d49ae6c70d1994ffd6b8cb38ccbcbc2aa4a6e66cc39228"
Jul 06 22:26:40.981083 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:40.981063    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8dc4771a516d0840c7d49ae6c70d1994ffd6b8cb38ccbcbc2aa4a6e66cc39228"} err="failed to get container status \"8dc4771a516d0840c7d49ae6c70d1994ffd6b8cb38ccbcbc2aa4a6e66cc39228\": rpc error: code = NotFound desc = could not find container \"8dc4771a516d0840c7d49ae6c70d1994ffd6b8cb38ccbcbc2aa4a6e66cc39228\": container with ID starting with 8dc4771a516d0840c7d49ae6c70d1994ffd6b8cb38ccbcbc2aa4a6e66cc39228 not found: ID does not exist"
Jul 06 22:26:41.079741 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:41.079713    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="858cf195-514b-452e-9d6a-57605def0790" path="/var/lib/kubelet/pods/858cf195-514b-452e-9d6a-57605def0790/volumes"
Jul 06 22:26:41.121955 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:41.121934    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/50dfcfd6-7883-4a00-ae83-75eda98d73f8-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"50dfcfd6-7883-4a00-ae83-75eda98d73f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 22:26:41.122088 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:41.121960    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/50dfcfd6-7883-4a00-ae83-75eda98d73f8-config\") pod \"prometheus-k8s-0\" (UID: \"50dfcfd6-7883-4a00-ae83-75eda98d73f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 22:26:41.122088 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:41.121977    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/50dfcfd6-7883-4a00-ae83-75eda98d73f8-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"50dfcfd6-7883-4a00-ae83-75eda98d73f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 22:26:41.122088 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:41.121998    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/50dfcfd6-7883-4a00-ae83-75eda98d73f8-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"50dfcfd6-7883-4a00-ae83-75eda98d73f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 22:26:41.122088 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:41.122013    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/50dfcfd6-7883-4a00-ae83-75eda98d73f8-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"50dfcfd6-7883-4a00-ae83-75eda98d73f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 22:26:41.122088 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:41.122039    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/50dfcfd6-7883-4a00-ae83-75eda98d73f8-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"50dfcfd6-7883-4a00-ae83-75eda98d73f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 22:26:41.122088 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:41.122058    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/50dfcfd6-7883-4a00-ae83-75eda98d73f8-config-out\") pod \"prometheus-k8s-0\" (UID: \"50dfcfd6-7883-4a00-ae83-75eda98d73f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 22:26:41.122088 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:41.122074    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/50dfcfd6-7883-4a00-ae83-75eda98d73f8-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"50dfcfd6-7883-4a00-ae83-75eda98d73f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 22:26:41.122088 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:41.122091    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/50dfcfd6-7883-4a00-ae83-75eda98d73f8-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"50dfcfd6-7883-4a00-ae83-75eda98d73f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 22:26:41.122443 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:41.122105    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/50dfcfd6-7883-4a00-ae83-75eda98d73f8-web-config\") pod \"prometheus-k8s-0\" (UID: \"50dfcfd6-7883-4a00-ae83-75eda98d73f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 22:26:41.122443 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:41.122128    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/50dfcfd6-7883-4a00-ae83-75eda98d73f8-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"50dfcfd6-7883-4a00-ae83-75eda98d73f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 22:26:41.122443 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:41.122213    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/50dfcfd6-7883-4a00-ae83-75eda98d73f8-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"50dfcfd6-7883-4a00-ae83-75eda98d73f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 22:26:41.122443 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:41.122253    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/50dfcfd6-7883-4a00-ae83-75eda98d73f8-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"50dfcfd6-7883-4a00-ae83-75eda98d73f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 22:26:41.122443 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:41.122276    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/50dfcfd6-7883-4a00-ae83-75eda98d73f8-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"50dfcfd6-7883-4a00-ae83-75eda98d73f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 22:26:41.122443 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:41.122294    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/50dfcfd6-7883-4a00-ae83-75eda98d73f8-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"50dfcfd6-7883-4a00-ae83-75eda98d73f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 22:26:41.122443 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:41.122313    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/50dfcfd6-7883-4a00-ae83-75eda98d73f8-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"50dfcfd6-7883-4a00-ae83-75eda98d73f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 22:26:41.122443 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:41.122332    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/50dfcfd6-7883-4a00-ae83-75eda98d73f8-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"50dfcfd6-7883-4a00-ae83-75eda98d73f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 22:26:41.122443 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:41.122348    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/50dfcfd6-7883-4a00-ae83-75eda98d73f8-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"50dfcfd6-7883-4a00-ae83-75eda98d73f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 22:26:41.122443 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:41.122423    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-flsxl\" (UniqueName: \"kubernetes.io/projected/50dfcfd6-7883-4a00-ae83-75eda98d73f8-kube-api-access-flsxl\") pod \"prometheus-k8s-0\" (UID: \"50dfcfd6-7883-4a00-ae83-75eda98d73f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 22:26:41.122780 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:41.122465    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/50dfcfd6-7883-4a00-ae83-75eda98d73f8-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"50dfcfd6-7883-4a00-ae83-75eda98d73f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 22:26:41.223402 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:41.223375    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/50dfcfd6-7883-4a00-ae83-75eda98d73f8-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"50dfcfd6-7883-4a00-ae83-75eda98d73f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 22:26:41.223548 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:41.223407    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/50dfcfd6-7883-4a00-ae83-75eda98d73f8-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"50dfcfd6-7883-4a00-ae83-75eda98d73f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 22:26:41.223548 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:41.223423    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/50dfcfd6-7883-4a00-ae83-75eda98d73f8-web-config\") pod \"prometheus-k8s-0\" (UID: \"50dfcfd6-7883-4a00-ae83-75eda98d73f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 22:26:41.223548 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:41.223442    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/50dfcfd6-7883-4a00-ae83-75eda98d73f8-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"50dfcfd6-7883-4a00-ae83-75eda98d73f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 22:26:41.223681 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:41.223606    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/50dfcfd6-7883-4a00-ae83-75eda98d73f8-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"50dfcfd6-7883-4a00-ae83-75eda98d73f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 22:26:41.223681 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:41.223658    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/50dfcfd6-7883-4a00-ae83-75eda98d73f8-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"50dfcfd6-7883-4a00-ae83-75eda98d73f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 22:26:41.223782 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:41.223695    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/50dfcfd6-7883-4a00-ae83-75eda98d73f8-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"50dfcfd6-7883-4a00-ae83-75eda98d73f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 22:26:41.223782 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:41.223724    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/50dfcfd6-7883-4a00-ae83-75eda98d73f8-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"50dfcfd6-7883-4a00-ae83-75eda98d73f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 22:26:41.223782 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:41.223759    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/50dfcfd6-7883-4a00-ae83-75eda98d73f8-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"50dfcfd6-7883-4a00-ae83-75eda98d73f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 22:26:41.223921 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:41.223796    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/50dfcfd6-7883-4a00-ae83-75eda98d73f8-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"50dfcfd6-7883-4a00-ae83-75eda98d73f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 22:26:41.223921 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:41.223826    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/50dfcfd6-7883-4a00-ae83-75eda98d73f8-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"50dfcfd6-7883-4a00-ae83-75eda98d73f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 22:26:41.223921 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:41.223866    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-flsxl\" (UniqueName: \"kubernetes.io/projected/50dfcfd6-7883-4a00-ae83-75eda98d73f8-kube-api-access-flsxl\") pod \"prometheus-k8s-0\" (UID: \"50dfcfd6-7883-4a00-ae83-75eda98d73f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 22:26:41.223921 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:41.223910    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/50dfcfd6-7883-4a00-ae83-75eda98d73f8-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"50dfcfd6-7883-4a00-ae83-75eda98d73f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 22:26:41.224094 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:41.223919    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/50dfcfd6-7883-4a00-ae83-75eda98d73f8-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"50dfcfd6-7883-4a00-ae83-75eda98d73f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 22:26:41.224094 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:41.223971    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/50dfcfd6-7883-4a00-ae83-75eda98d73f8-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"50dfcfd6-7883-4a00-ae83-75eda98d73f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 22:26:41.224094 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:41.224002    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/50dfcfd6-7883-4a00-ae83-75eda98d73f8-config\") pod \"prometheus-k8s-0\" (UID: \"50dfcfd6-7883-4a00-ae83-75eda98d73f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 22:26:41.224094 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:41.224029    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/50dfcfd6-7883-4a00-ae83-75eda98d73f8-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"50dfcfd6-7883-4a00-ae83-75eda98d73f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 22:26:41.224094 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:41.224064    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/50dfcfd6-7883-4a00-ae83-75eda98d73f8-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"50dfcfd6-7883-4a00-ae83-75eda98d73f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 22:26:41.224094 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:41.224089    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/50dfcfd6-7883-4a00-ae83-75eda98d73f8-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"50dfcfd6-7883-4a00-ae83-75eda98d73f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 22:26:41.224315 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:41.224095    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/50dfcfd6-7883-4a00-ae83-75eda98d73f8-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"50dfcfd6-7883-4a00-ae83-75eda98d73f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 22:26:41.224315 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:41.224120    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/50dfcfd6-7883-4a00-ae83-75eda98d73f8-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"50dfcfd6-7883-4a00-ae83-75eda98d73f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 22:26:41.224315 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:41.224151    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/50dfcfd6-7883-4a00-ae83-75eda98d73f8-config-out\") pod \"prometheus-k8s-0\" (UID: \"50dfcfd6-7883-4a00-ae83-75eda98d73f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 22:26:41.224315 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:41.224249    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/50dfcfd6-7883-4a00-ae83-75eda98d73f8-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"50dfcfd6-7883-4a00-ae83-75eda98d73f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 22:26:41.225514 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:41.224769    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/50dfcfd6-7883-4a00-ae83-75eda98d73f8-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"50dfcfd6-7883-4a00-ae83-75eda98d73f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 22:26:41.225514 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:41.225186    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/50dfcfd6-7883-4a00-ae83-75eda98d73f8-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"50dfcfd6-7883-4a00-ae83-75eda98d73f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 22:26:41.225904 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:41.225880    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/50dfcfd6-7883-4a00-ae83-75eda98d73f8-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"50dfcfd6-7883-4a00-ae83-75eda98d73f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 22:26:41.227274 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:41.226937    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/50dfcfd6-7883-4a00-ae83-75eda98d73f8-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"50dfcfd6-7883-4a00-ae83-75eda98d73f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 22:26:41.227274 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:41.226944    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/50dfcfd6-7883-4a00-ae83-75eda98d73f8-web-config\") pod \"prometheus-k8s-0\" (UID: \"50dfcfd6-7883-4a00-ae83-75eda98d73f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 22:26:41.227274 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:41.226970    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/50dfcfd6-7883-4a00-ae83-75eda98d73f8-config-out\") pod \"prometheus-k8s-0\" (UID: \"50dfcfd6-7883-4a00-ae83-75eda98d73f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 22:26:41.227274 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:41.227156    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/50dfcfd6-7883-4a00-ae83-75eda98d73f8-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"50dfcfd6-7883-4a00-ae83-75eda98d73f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 22:26:41.227773 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:41.227404    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/50dfcfd6-7883-4a00-ae83-75eda98d73f8-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"50dfcfd6-7883-4a00-ae83-75eda98d73f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 22:26:41.228274 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:41.228125    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/50dfcfd6-7883-4a00-ae83-75eda98d73f8-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"50dfcfd6-7883-4a00-ae83-75eda98d73f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 22:26:41.228274 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:41.228214    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/50dfcfd6-7883-4a00-ae83-75eda98d73f8-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"50dfcfd6-7883-4a00-ae83-75eda98d73f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 22:26:41.228429 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:41.228379    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/50dfcfd6-7883-4a00-ae83-75eda98d73f8-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"50dfcfd6-7883-4a00-ae83-75eda98d73f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 22:26:41.228708 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:41.228687    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/50dfcfd6-7883-4a00-ae83-75eda98d73f8-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"50dfcfd6-7883-4a00-ae83-75eda98d73f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 22:26:41.229215 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:41.229198    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/50dfcfd6-7883-4a00-ae83-75eda98d73f8-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"50dfcfd6-7883-4a00-ae83-75eda98d73f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 22:26:41.229690 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:41.229672    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/50dfcfd6-7883-4a00-ae83-75eda98d73f8-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"50dfcfd6-7883-4a00-ae83-75eda98d73f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 22:26:41.229763 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:41.229690    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/50dfcfd6-7883-4a00-ae83-75eda98d73f8-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"50dfcfd6-7883-4a00-ae83-75eda98d73f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 22:26:41.230009 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:41.229989    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/50dfcfd6-7883-4a00-ae83-75eda98d73f8-config\") pod \"prometheus-k8s-0\" (UID: \"50dfcfd6-7883-4a00-ae83-75eda98d73f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 22:26:41.233045 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:41.233028    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-flsxl\" (UniqueName: \"kubernetes.io/projected/50dfcfd6-7883-4a00-ae83-75eda98d73f8-kube-api-access-flsxl\") pod \"prometheus-k8s-0\" (UID: \"50dfcfd6-7883-4a00-ae83-75eda98d73f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 22:26:41.278648 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:41.278622    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 22:26:41.412975 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:41.412944    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 06 22:26:41.415722 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:26:41.415697    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50dfcfd6_7883_4a00_ae83_75eda98d73f8.slice/crio-8420d75524ebe5b1add34e0f7d3201c26907528461af697118dbb4ff3e9274d2 WatchSource:0}: Error finding container 8420d75524ebe5b1add34e0f7d3201c26907528461af697118dbb4ff3e9274d2: Status 404 returned error can't find the container with id 8420d75524ebe5b1add34e0f7d3201c26907528461af697118dbb4ff3e9274d2
Jul 06 22:26:41.912056 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:41.911967    2563 generic.go:356] "Generic (PLEG): container finished" podID="50dfcfd6-7883-4a00-ae83-75eda98d73f8" containerID="166fc69456fc97e1ab698158b039f01eaacbbad941df6cc15a8e6f4b98806b1e" exitCode=0
Jul 06 22:26:41.912056 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:41.912023    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"50dfcfd6-7883-4a00-ae83-75eda98d73f8","Type":"ContainerDied","Data":"166fc69456fc97e1ab698158b039f01eaacbbad941df6cc15a8e6f4b98806b1e"}
Jul 06 22:26:41.912444 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:41.912070    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"50dfcfd6-7883-4a00-ae83-75eda98d73f8","Type":"ContainerStarted","Data":"8420d75524ebe5b1add34e0f7d3201c26907528461af697118dbb4ff3e9274d2"}
Jul 06 22:26:42.919744 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:42.919703    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"50dfcfd6-7883-4a00-ae83-75eda98d73f8","Type":"ContainerStarted","Data":"5ee536ea25d229080ff403466c7adb98e3affcd6ac06c33abb65e361cce8eb2d"}
Jul 06 22:26:42.920133 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:42.919803    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"50dfcfd6-7883-4a00-ae83-75eda98d73f8","Type":"ContainerStarted","Data":"3d0ed46834c124aac6d5b313872c83a94b5f3a255aaab5ad14563520640aa4dc"}
Jul 06 22:26:42.920133 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:42.919846    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"50dfcfd6-7883-4a00-ae83-75eda98d73f8","Type":"ContainerStarted","Data":"cc97372f616a2d94c3eac1e2066fb737fa4bed489c484b5a24aaa98966a9b763"}
Jul 06 22:26:42.920133 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:42.919874    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"50dfcfd6-7883-4a00-ae83-75eda98d73f8","Type":"ContainerStarted","Data":"9073ebd51344f3caf4e5ed7bd2616879aa5429bef1b90d382b4d96a735d4bd5b"}
Jul 06 22:26:42.920133 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:42.919898    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"50dfcfd6-7883-4a00-ae83-75eda98d73f8","Type":"ContainerStarted","Data":"4cbffa6846c213b9c98057b2099aefbf82d8833fa619bf5f5bc882dbea3e8f8b"}
Jul 06 22:26:42.920133 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:42.919931    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"50dfcfd6-7883-4a00-ae83-75eda98d73f8","Type":"ContainerStarted","Data":"72807c9a90f42d0caf2e42b630e032fbebdb3cd396122fba1d51ae417d38e97b"}
Jul 06 22:26:42.948838 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:42.948756    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.929571945 podStartE2EDuration="2.948744051s" totalImagesPullingTime="0s" totalInitContainerRuntime="19.172106ms" isStatefulPod=false podCreationTimestamp="2026-07-06 22:26:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:26:42.947566308 +0000 UTC m=+168.372612850" watchObservedRunningTime="2026-07-06 22:26:42.948744051 +0000 UTC m=+168.373790592"
Jul 06 22:26:46.279299 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:26:46.279215    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 22:27:41.279653 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:27:41.279614    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 22:27:41.295604 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:27:41.295578    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 22:27:42.148037 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:27:42.148007    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 06 22:28:54.987666 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:28:54.987638    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-t6jpk_03309f33-ff47-447d-956f-c89bdd4f8054/ovn-acl-logging/0.log"
Jul 06 22:28:54.991917 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:28:54.991891    2563 kubelet.go:1648] "Image garbage collection succeeded"
Jul 06 22:28:54.994590 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:28:54.994572    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-t6jpk_03309f33-ff47-447d-956f-c89bdd4f8054/ovn-acl-logging/0.log"
Jul 06 22:29:42.482025 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:29:42.481990    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-85fcf54485-9lvpp"]
Jul 06 22:29:42.484326 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:29:42.484309    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-85fcf54485-9lvpp"
Jul 06 22:29:42.489121 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:29:42.489103    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 06 22:29:42.489410 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:29:42.489394    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 06 22:29:42.489676 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:29:42.489657    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 06 22:29:42.489954 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:29:42.489938    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 06 22:29:42.490196 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:29:42.490162    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-nd4qr\""
Jul 06 22:29:42.509697 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:29:42.509673    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-85fcf54485-9lvpp"]
Jul 06 22:29:42.521329 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:29:42.521302    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e4aeb198-110f-4a6e-99dc-d3df527d8fc0-apiservice-cert\") pod \"opendatahub-operator-controller-manager-85fcf54485-9lvpp\" (UID: \"e4aeb198-110f-4a6e-99dc-d3df527d8fc0\") " pod="opendatahub/opendatahub-operator-controller-manager-85fcf54485-9lvpp"
Jul 06 22:29:42.521458 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:29:42.521337    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7rh85\" (UniqueName: \"kubernetes.io/projected/e4aeb198-110f-4a6e-99dc-d3df527d8fc0-kube-api-access-7rh85\") pod \"opendatahub-operator-controller-manager-85fcf54485-9lvpp\" (UID: \"e4aeb198-110f-4a6e-99dc-d3df527d8fc0\") " pod="opendatahub/opendatahub-operator-controller-manager-85fcf54485-9lvpp"
Jul 06 22:29:42.521458 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:29:42.521355    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e4aeb198-110f-4a6e-99dc-d3df527d8fc0-webhook-cert\") pod \"opendatahub-operator-controller-manager-85fcf54485-9lvpp\" (UID: \"e4aeb198-110f-4a6e-99dc-d3df527d8fc0\") " pod="opendatahub/opendatahub-operator-controller-manager-85fcf54485-9lvpp"
Jul 06 22:29:42.521458 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:29:42.521413    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e4aeb198-110f-4a6e-99dc-d3df527d8fc0-tmp\") pod \"opendatahub-operator-controller-manager-85fcf54485-9lvpp\" (UID: \"e4aeb198-110f-4a6e-99dc-d3df527d8fc0\") " pod="opendatahub/opendatahub-operator-controller-manager-85fcf54485-9lvpp"
Jul 06 22:29:42.521579 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:29:42.521510    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e4aeb198-110f-4a6e-99dc-d3df527d8fc0-manifests\") pod \"opendatahub-operator-controller-manager-85fcf54485-9lvpp\" (UID: \"e4aeb198-110f-4a6e-99dc-d3df527d8fc0\") " pod="opendatahub/opendatahub-operator-controller-manager-85fcf54485-9lvpp"
Jul 06 22:29:42.622344 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:29:42.622312    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e4aeb198-110f-4a6e-99dc-d3df527d8fc0-apiservice-cert\") pod \"opendatahub-operator-controller-manager-85fcf54485-9lvpp\" (UID: \"e4aeb198-110f-4a6e-99dc-d3df527d8fc0\") " pod="opendatahub/opendatahub-operator-controller-manager-85fcf54485-9lvpp"
Jul 06 22:29:42.622344 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:29:42.622351    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7rh85\" (UniqueName: \"kubernetes.io/projected/e4aeb198-110f-4a6e-99dc-d3df527d8fc0-kube-api-access-7rh85\") pod \"opendatahub-operator-controller-manager-85fcf54485-9lvpp\" (UID: \"e4aeb198-110f-4a6e-99dc-d3df527d8fc0\") " pod="opendatahub/opendatahub-operator-controller-manager-85fcf54485-9lvpp"
Jul 06 22:29:42.622619 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:29:42.622368    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e4aeb198-110f-4a6e-99dc-d3df527d8fc0-webhook-cert\") pod \"opendatahub-operator-controller-manager-85fcf54485-9lvpp\" (UID: \"e4aeb198-110f-4a6e-99dc-d3df527d8fc0\") " pod="opendatahub/opendatahub-operator-controller-manager-85fcf54485-9lvpp"
Jul 06 22:29:42.622619 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:29:42.622505    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e4aeb198-110f-4a6e-99dc-d3df527d8fc0-tmp\") pod \"opendatahub-operator-controller-manager-85fcf54485-9lvpp\" (UID: \"e4aeb198-110f-4a6e-99dc-d3df527d8fc0\") " pod="opendatahub/opendatahub-operator-controller-manager-85fcf54485-9lvpp"
Jul 06 22:29:42.622619 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:29:42.622565    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e4aeb198-110f-4a6e-99dc-d3df527d8fc0-manifests\") pod \"opendatahub-operator-controller-manager-85fcf54485-9lvpp\" (UID: \"e4aeb198-110f-4a6e-99dc-d3df527d8fc0\") " pod="opendatahub/opendatahub-operator-controller-manager-85fcf54485-9lvpp"
Jul 06 22:29:42.623196 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:29:42.623097    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e4aeb198-110f-4a6e-99dc-d3df527d8fc0-manifests\") pod \"opendatahub-operator-controller-manager-85fcf54485-9lvpp\" (UID: \"e4aeb198-110f-4a6e-99dc-d3df527d8fc0\") " pod="opendatahub/opendatahub-operator-controller-manager-85fcf54485-9lvpp"
Jul 06 22:29:42.625093 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:29:42.625066    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e4aeb198-110f-4a6e-99dc-d3df527d8fc0-tmp\") pod \"opendatahub-operator-controller-manager-85fcf54485-9lvpp\" (UID: \"e4aeb198-110f-4a6e-99dc-d3df527d8fc0\") " pod="opendatahub/opendatahub-operator-controller-manager-85fcf54485-9lvpp"
Jul 06 22:29:42.625292 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:29:42.625268    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e4aeb198-110f-4a6e-99dc-d3df527d8fc0-apiservice-cert\") pod \"opendatahub-operator-controller-manager-85fcf54485-9lvpp\" (UID: \"e4aeb198-110f-4a6e-99dc-d3df527d8fc0\") " pod="opendatahub/opendatahub-operator-controller-manager-85fcf54485-9lvpp"
Jul 06 22:29:42.625385 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:29:42.625312    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e4aeb198-110f-4a6e-99dc-d3df527d8fc0-webhook-cert\") pod \"opendatahub-operator-controller-manager-85fcf54485-9lvpp\" (UID: \"e4aeb198-110f-4a6e-99dc-d3df527d8fc0\") " pod="opendatahub/opendatahub-operator-controller-manager-85fcf54485-9lvpp"
Jul 06 22:29:42.639645 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:29:42.639622    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7rh85\" (UniqueName: \"kubernetes.io/projected/e4aeb198-110f-4a6e-99dc-d3df527d8fc0-kube-api-access-7rh85\") pod \"opendatahub-operator-controller-manager-85fcf54485-9lvpp\" (UID: \"e4aeb198-110f-4a6e-99dc-d3df527d8fc0\") " pod="opendatahub/opendatahub-operator-controller-manager-85fcf54485-9lvpp"
Jul 06 22:29:42.794576 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:29:42.794481    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-85fcf54485-9lvpp"
Jul 06 22:29:42.938440 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:29:42.938393    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-85fcf54485-9lvpp"]
Jul 06 22:29:42.941516 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:29:42.941490    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4aeb198_110f_4a6e_99dc_d3df527d8fc0.slice/crio-e413abac4df24017f52dab5a73aaa65c0af34d5c90851283399a18bc8f6ba275 WatchSource:0}: Error finding container e413abac4df24017f52dab5a73aaa65c0af34d5c90851283399a18bc8f6ba275: Status 404 returned error can't find the container with id e413abac4df24017f52dab5a73aaa65c0af34d5c90851283399a18bc8f6ba275
Jul 06 22:29:42.943699 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:29:42.943683    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 06 22:29:43.574365 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:29:43.574324    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-85fcf54485-9lvpp" event={"ID":"e4aeb198-110f-4a6e-99dc-d3df527d8fc0","Type":"ContainerStarted","Data":"e413abac4df24017f52dab5a73aaa65c0af34d5c90851283399a18bc8f6ba275"}
Jul 06 22:29:46.588829 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:29:46.588792    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-85fcf54485-9lvpp" event={"ID":"e4aeb198-110f-4a6e-99dc-d3df527d8fc0","Type":"ContainerStarted","Data":"2d53d2e58bbacd44dc016361294b35401ce5fc0ce049c0dfbadea0991119a77b"}
Jul 06 22:29:47.594505 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:29:47.594471    2563 generic.go:356] "Generic (PLEG): container finished" podID="e4aeb198-110f-4a6e-99dc-d3df527d8fc0" containerID="2d53d2e58bbacd44dc016361294b35401ce5fc0ce049c0dfbadea0991119a77b" exitCode=0
Jul 06 22:29:47.594892 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:29:47.594519    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-85fcf54485-9lvpp" event={"ID":"e4aeb198-110f-4a6e-99dc-d3df527d8fc0","Type":"ContainerDied","Data":"2d53d2e58bbacd44dc016361294b35401ce5fc0ce049c0dfbadea0991119a77b"}
Jul 06 22:29:48.599957 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:29:48.599921    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-85fcf54485-9lvpp" event={"ID":"e4aeb198-110f-4a6e-99dc-d3df527d8fc0","Type":"ContainerStarted","Data":"4fb552d509efebae517245efc03140091619072c54c16596633ab79882e4383d"}
Jul 06 22:29:48.600397 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:29:48.600069    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-85fcf54485-9lvpp"
Jul 06 22:29:48.629546 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:29:48.629489    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-85fcf54485-9lvpp" podStartSLOduration=2.384897446 podStartE2EDuration="6.629470086s" totalImagesPullingTime="3.172256877s" totalInitContainerRuntime="1.072315763s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:29:42 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:29:48.628375191 +0000 UTC m=+354.053421731" watchObservedRunningTime="2026-07-06 22:29:48.629470086 +0000 UTC m=+354.054516624"
Jul 06 22:29:59.606134 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:29:59.606101    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-85fcf54485-9lvpp"
Jul 06 22:30:03.078951 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:30:03.078923    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-r5zbk"]
Jul 06 22:30:03.089614 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:30:03.089584    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-r5zbk"
Jul 06 22:30:03.091111 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:30:03.091084    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-r5zbk"]
Jul 06 22:30:03.092153 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:30:03.092129    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Jul 06 22:30:03.092302 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:30:03.092152    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-vzp55\""
Jul 06 22:30:03.200700 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:30:03.200665    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ffd77de9-663b-495f-9b8d-d744c8145a03-tls-certs\") pod \"model-serving-api-54d89bd5fc-r5zbk\" (UID: \"ffd77de9-663b-495f-9b8d-d744c8145a03\") " pod="opendatahub/model-serving-api-54d89bd5fc-r5zbk"
Jul 06 22:30:03.200855 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:30:03.200729    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c77vr\" (UniqueName: \"kubernetes.io/projected/ffd77de9-663b-495f-9b8d-d744c8145a03-kube-api-access-c77vr\") pod \"model-serving-api-54d89bd5fc-r5zbk\" (UID: \"ffd77de9-663b-495f-9b8d-d744c8145a03\") " pod="opendatahub/model-serving-api-54d89bd5fc-r5zbk"
Jul 06 22:30:03.275430 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:30:03.275395    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-7f6975f4cc-hmkl4"]
Jul 06 22:30:03.278377 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:30:03.278351    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7f6975f4cc-hmkl4"
Jul 06 22:30:03.281517 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:30:03.281095    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 06 22:30:03.281517 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:30:03.281402    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 06 22:30:03.281738 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:30:03.281542    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-zd6q2\""
Jul 06 22:30:03.281738 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:30:03.281549    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 06 22:30:03.281984 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:30:03.281963    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 06 22:30:03.290584 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:30:03.290562    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7f6975f4cc-hmkl4"]
Jul 06 22:30:03.301847 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:30:03.301826    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c77vr\" (UniqueName: \"kubernetes.io/projected/ffd77de9-663b-495f-9b8d-d744c8145a03-kube-api-access-c77vr\") pod \"model-serving-api-54d89bd5fc-r5zbk\" (UID: \"ffd77de9-663b-495f-9b8d-d744c8145a03\") " pod="opendatahub/model-serving-api-54d89bd5fc-r5zbk"
Jul 06 22:30:03.301962 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:30:03.301873    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ffd77de9-663b-495f-9b8d-d744c8145a03-tls-certs\") pod \"model-serving-api-54d89bd5fc-r5zbk\" (UID: \"ffd77de9-663b-495f-9b8d-d744c8145a03\") " pod="opendatahub/model-serving-api-54d89bd5fc-r5zbk"
Jul 06 22:30:03.302021 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:30:03.301980    2563 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found
Jul 06 22:30:03.302073 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:30:03.302027    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ffd77de9-663b-495f-9b8d-d744c8145a03-tls-certs podName:ffd77de9-663b-495f-9b8d-d744c8145a03 nodeName:}" failed. No retries permitted until 2026-07-06 22:30:03.802012639 +0000 UTC m=+369.227059158 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/ffd77de9-663b-495f-9b8d-d744c8145a03-tls-certs") pod "model-serving-api-54d89bd5fc-r5zbk" (UID: "ffd77de9-663b-495f-9b8d-d744c8145a03") : secret "model-serving-api-tls" not found
Jul 06 22:30:03.312903 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:30:03.312878    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c77vr\" (UniqueName: \"kubernetes.io/projected/ffd77de9-663b-495f-9b8d-d744c8145a03-kube-api-access-c77vr\") pod \"model-serving-api-54d89bd5fc-r5zbk\" (UID: \"ffd77de9-663b-495f-9b8d-d744c8145a03\") " pod="opendatahub/model-serving-api-54d89bd5fc-r5zbk"
Jul 06 22:30:03.403010 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:30:03.402908    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s2d7v\" (UniqueName: \"kubernetes.io/projected/bfd0e5f4-30d1-4e01-acb1-e6f254227584-kube-api-access-s2d7v\") pod \"kube-auth-proxy-7f6975f4cc-hmkl4\" (UID: \"bfd0e5f4-30d1-4e01-acb1-e6f254227584\") " pod="openshift-ingress/kube-auth-proxy-7f6975f4cc-hmkl4"
Jul 06 22:30:03.403010 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:30:03.402956    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bfd0e5f4-30d1-4e01-acb1-e6f254227584-tmp\") pod \"kube-auth-proxy-7f6975f4cc-hmkl4\" (UID: \"bfd0e5f4-30d1-4e01-acb1-e6f254227584\") " pod="openshift-ingress/kube-auth-proxy-7f6975f4cc-hmkl4"
Jul 06 22:30:03.403261 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:30:03.403034    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bfd0e5f4-30d1-4e01-acb1-e6f254227584-tls-certs\") pod \"kube-auth-proxy-7f6975f4cc-hmkl4\" (UID: \"bfd0e5f4-30d1-4e01-acb1-e6f254227584\") " pod="openshift-ingress/kube-auth-proxy-7f6975f4cc-hmkl4"
Jul 06 22:30:03.504319 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:30:03.504289    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s2d7v\" (UniqueName: \"kubernetes.io/projected/bfd0e5f4-30d1-4e01-acb1-e6f254227584-kube-api-access-s2d7v\") pod \"kube-auth-proxy-7f6975f4cc-hmkl4\" (UID: \"bfd0e5f4-30d1-4e01-acb1-e6f254227584\") " pod="openshift-ingress/kube-auth-proxy-7f6975f4cc-hmkl4"
Jul 06 22:30:03.504486 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:30:03.504330    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bfd0e5f4-30d1-4e01-acb1-e6f254227584-tmp\") pod \"kube-auth-proxy-7f6975f4cc-hmkl4\" (UID: \"bfd0e5f4-30d1-4e01-acb1-e6f254227584\") " pod="openshift-ingress/kube-auth-proxy-7f6975f4cc-hmkl4"
Jul 06 22:30:03.504486 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:30:03.504359    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bfd0e5f4-30d1-4e01-acb1-e6f254227584-tls-certs\") pod \"kube-auth-proxy-7f6975f4cc-hmkl4\" (UID: \"bfd0e5f4-30d1-4e01-acb1-e6f254227584\") " pod="openshift-ingress/kube-auth-proxy-7f6975f4cc-hmkl4"
Jul 06 22:30:03.506672 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:30:03.506645    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bfd0e5f4-30d1-4e01-acb1-e6f254227584-tmp\") pod \"kube-auth-proxy-7f6975f4cc-hmkl4\" (UID: \"bfd0e5f4-30d1-4e01-acb1-e6f254227584\") " pod="openshift-ingress/kube-auth-proxy-7f6975f4cc-hmkl4"
Jul 06 22:30:03.506901 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:30:03.506882    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bfd0e5f4-30d1-4e01-acb1-e6f254227584-tls-certs\") pod \"kube-auth-proxy-7f6975f4cc-hmkl4\" (UID: \"bfd0e5f4-30d1-4e01-acb1-e6f254227584\") " pod="openshift-ingress/kube-auth-proxy-7f6975f4cc-hmkl4"
Jul 06 22:30:03.512022 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:30:03.512002    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s2d7v\" (UniqueName: \"kubernetes.io/projected/bfd0e5f4-30d1-4e01-acb1-e6f254227584-kube-api-access-s2d7v\") pod \"kube-auth-proxy-7f6975f4cc-hmkl4\" (UID: \"bfd0e5f4-30d1-4e01-acb1-e6f254227584\") " pod="openshift-ingress/kube-auth-proxy-7f6975f4cc-hmkl4"
Jul 06 22:30:03.590353 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:30:03.590316    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7f6975f4cc-hmkl4"
Jul 06 22:30:03.737034 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:30:03.736971    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7f6975f4cc-hmkl4"]
Jul 06 22:30:03.741901 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:30:03.741845    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbfd0e5f4_30d1_4e01_acb1_e6f254227584.slice/crio-1be6a9af3492ea5802c43bdacd06031e5b1444897575de93f128ec4a6a53a651 WatchSource:0}: Error finding container 1be6a9af3492ea5802c43bdacd06031e5b1444897575de93f128ec4a6a53a651: Status 404 returned error can't find the container with id 1be6a9af3492ea5802c43bdacd06031e5b1444897575de93f128ec4a6a53a651
Jul 06 22:30:03.806421 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:30:03.806383    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ffd77de9-663b-495f-9b8d-d744c8145a03-tls-certs\") pod \"model-serving-api-54d89bd5fc-r5zbk\" (UID: \"ffd77de9-663b-495f-9b8d-d744c8145a03\") " pod="opendatahub/model-serving-api-54d89bd5fc-r5zbk"
Jul 06 22:30:03.808875 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:30:03.808857    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ffd77de9-663b-495f-9b8d-d744c8145a03-tls-certs\") pod \"model-serving-api-54d89bd5fc-r5zbk\" (UID: \"ffd77de9-663b-495f-9b8d-d744c8145a03\") " pod="opendatahub/model-serving-api-54d89bd5fc-r5zbk"
Jul 06 22:30:04.005616 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:30:04.005580    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-r5zbk"
Jul 06 22:30:04.138240 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:30:04.138050    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-r5zbk"]
Jul 06 22:30:04.140884 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:30:04.140848    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podffd77de9_663b_495f_9b8d_d744c8145a03.slice/crio-70a2916df707c388efda74da371d83c040ee35a2d2a6e878795304905b3eb69d WatchSource:0}: Error finding container 70a2916df707c388efda74da371d83c040ee35a2d2a6e878795304905b3eb69d: Status 404 returned error can't find the container with id 70a2916df707c388efda74da371d83c040ee35a2d2a6e878795304905b3eb69d
Jul 06 22:30:04.665008 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:30:04.664967    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7f6975f4cc-hmkl4" event={"ID":"bfd0e5f4-30d1-4e01-acb1-e6f254227584","Type":"ContainerStarted","Data":"1be6a9af3492ea5802c43bdacd06031e5b1444897575de93f128ec4a6a53a651"}
Jul 06 22:30:04.666608 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:30:04.666554    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-r5zbk" event={"ID":"ffd77de9-663b-495f-9b8d-d744c8145a03","Type":"ContainerStarted","Data":"70a2916df707c388efda74da371d83c040ee35a2d2a6e878795304905b3eb69d"}
Jul 06 22:30:06.680732 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:30:06.680649    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7f6975f4cc-hmkl4" event={"ID":"bfd0e5f4-30d1-4e01-acb1-e6f254227584","Type":"ContainerStarted","Data":"259079e04b5c8709b21e5fd9e7b4ea56ee3c08ef730c004d354b6a3dbd992582"}
Jul 06 22:30:06.682145 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:30:06.682120    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-r5zbk" event={"ID":"ffd77de9-663b-495f-9b8d-d744c8145a03","Type":"ContainerStarted","Data":"d1d5d3ed8d34e502609ed46d3e37f4140fa1a3560e18799ed3a57d107b5b39df"}
Jul 06 22:30:06.682270 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:30:06.682215    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-r5zbk"
Jul 06 22:30:06.701912 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:30:06.701862    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-7f6975f4cc-hmkl4" podStartSLOduration=1.163706724 podStartE2EDuration="3.701843838s" totalImagesPullingTime="2.538137114s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:30:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:30:06.700614947 +0000 UTC m=+372.125661490" watchObservedRunningTime="2026-07-06 22:30:06.701843838 +0000 UTC m=+372.126890380"
Jul 06 22:30:06.725299 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:30:06.725243    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-r5zbk" podStartSLOduration=1.5857856350000001 podStartE2EDuration="3.725225884s" totalImagesPullingTime="2.139440249s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:30:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:30:06.723755191 +0000 UTC m=+372.148801746" watchObservedRunningTime="2026-07-06 22:30:06.725225884 +0000 UTC m=+372.150272424"
Jul 06 22:30:10.851128 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:30:10.851086    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-z2tm5"]
Jul 06 22:30:10.855599 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:30:10.855574    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-z2tm5"
Jul 06 22:30:10.860468 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:30:10.859592    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 06 22:30:10.860468 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:30:10.859785    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 06 22:30:10.860468 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:30:10.859925    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 06 22:30:10.860468 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:30:10.860123    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-nkt4c\""
Jul 06 22:30:10.860468 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:30:10.860300    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 06 22:30:10.886142 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:30:10.886106    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-z2tm5"]
Jul 06 22:30:10.972968 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:30:10.972937    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/8fe2c7e3-c0da-4480-90f8-d17046019fdd-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-z2tm5\" (UID: \"8fe2c7e3-c0da-4480-90f8-d17046019fdd\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-z2tm5"
Jul 06 22:30:10.973139 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:30:10.972986    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/8fe2c7e3-c0da-4480-90f8-d17046019fdd-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-z2tm5\" (UID: \"8fe2c7e3-c0da-4480-90f8-d17046019fdd\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-z2tm5"
Jul 06 22:30:10.973139 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:30:10.973037    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/8fe2c7e3-c0da-4480-90f8-d17046019fdd-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-z2tm5\" (UID: \"8fe2c7e3-c0da-4480-90f8-d17046019fdd\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-z2tm5"
Jul 06 22:30:10.973139 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:30:10.973061    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8fe2c7e3-c0da-4480-90f8-d17046019fdd-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-z2tm5\" (UID: \"8fe2c7e3-c0da-4480-90f8-d17046019fdd\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-z2tm5"
Jul 06 22:30:10.973139 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:30:10.973096    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zb8q4\" (UniqueName: \"kubernetes.io/projected/8fe2c7e3-c0da-4480-90f8-d17046019fdd-kube-api-access-zb8q4\") pod \"istiod-openshift-gateway-94bb8fbfd-z2tm5\" (UID: \"8fe2c7e3-c0da-4480-90f8-d17046019fdd\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-z2tm5"
Jul 06 22:30:10.973380 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:30:10.973156    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8fe2c7e3-c0da-4480-90f8-d17046019fdd-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-z2tm5\" (UID: \"8fe2c7e3-c0da-4480-90f8-d17046019fdd\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-z2tm5"
Jul 06 22:30:10.973380 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:30:10.973217    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/8fe2c7e3-c0da-4480-90f8-d17046019fdd-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-z2tm5\" (UID: \"8fe2c7e3-c0da-4480-90f8-d17046019fdd\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-z2tm5"
Jul 06 22:30:11.073716 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:30:11.073676    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8fe2c7e3-c0da-4480-90f8-d17046019fdd-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-z2tm5\" (UID: \"8fe2c7e3-c0da-4480-90f8-d17046019fdd\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-z2tm5"
Jul 06 22:30:11.073880 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:30:11.073732    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/8fe2c7e3-c0da-4480-90f8-d17046019fdd-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-z2tm5\" (UID: \"8fe2c7e3-c0da-4480-90f8-d17046019fdd\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-z2tm5"
Jul 06 22:30:11.073880 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:30:11.073789    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/8fe2c7e3-c0da-4480-90f8-d17046019fdd-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-z2tm5\" (UID: \"8fe2c7e3-c0da-4480-90f8-d17046019fdd\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-z2tm5"
Jul 06 22:30:11.073880 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:30:11.073839    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/8fe2c7e3-c0da-4480-90f8-d17046019fdd-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-z2tm5\" (UID: \"8fe2c7e3-c0da-4480-90f8-d17046019fdd\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-z2tm5"
Jul 06 22:30:11.073880 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:30:11.073872    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/8fe2c7e3-c0da-4480-90f8-d17046019fdd-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-z2tm5\" (UID: \"8fe2c7e3-c0da-4480-90f8-d17046019fdd\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-z2tm5"
Jul 06 22:30:11.074105 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:30:11.073971    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8fe2c7e3-c0da-4480-90f8-d17046019fdd-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-z2tm5\" (UID: \"8fe2c7e3-c0da-4480-90f8-d17046019fdd\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-z2tm5"
Jul 06 22:30:11.074105 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:30:11.074011    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zb8q4\" (UniqueName: \"kubernetes.io/projected/8fe2c7e3-c0da-4480-90f8-d17046019fdd-kube-api-access-zb8q4\") pod \"istiod-openshift-gateway-94bb8fbfd-z2tm5\" (UID: \"8fe2c7e3-c0da-4480-90f8-d17046019fdd\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-z2tm5"
Jul 06 22:30:11.075305 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:30:11.075276    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/8fe2c7e3-c0da-4480-90f8-d17046019fdd-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-z2tm5\" (UID: \"8fe2c7e3-c0da-4480-90f8-d17046019fdd\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-z2tm5"
Jul 06 22:30:11.076681 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:30:11.076655    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8fe2c7e3-c0da-4480-90f8-d17046019fdd-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-z2tm5\" (UID: \"8fe2c7e3-c0da-4480-90f8-d17046019fdd\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-z2tm5"
Jul 06 22:30:11.076787 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:30:11.076743    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/8fe2c7e3-c0da-4480-90f8-d17046019fdd-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-z2tm5\" (UID: \"8fe2c7e3-c0da-4480-90f8-d17046019fdd\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-z2tm5"
Jul 06 22:30:11.076849 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:30:11.076813    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/8fe2c7e3-c0da-4480-90f8-d17046019fdd-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-z2tm5\" (UID: \"8fe2c7e3-c0da-4480-90f8-d17046019fdd\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-z2tm5"
Jul 06 22:30:11.077058 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:30:11.077040    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/8fe2c7e3-c0da-4480-90f8-d17046019fdd-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-z2tm5\" (UID: \"8fe2c7e3-c0da-4480-90f8-d17046019fdd\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-z2tm5"
Jul 06 22:30:11.084505 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:30:11.084481    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8fe2c7e3-c0da-4480-90f8-d17046019fdd-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-z2tm5\" (UID: \"8fe2c7e3-c0da-4480-90f8-d17046019fdd\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-z2tm5"
Jul 06 22:30:11.084873 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:30:11.084855    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zb8q4\" (UniqueName: \"kubernetes.io/projected/8fe2c7e3-c0da-4480-90f8-d17046019fdd-kube-api-access-zb8q4\") pod \"istiod-openshift-gateway-94bb8fbfd-z2tm5\" (UID: \"8fe2c7e3-c0da-4480-90f8-d17046019fdd\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-z2tm5"
Jul 06 22:30:11.170760 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:30:11.170674    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-z2tm5"
Jul 06 22:30:11.313523 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:30:11.313493    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-z2tm5"]
Jul 06 22:30:11.316370 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:30:11.316344    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8fe2c7e3_c0da_4480_90f8_d17046019fdd.slice/crio-da0ce5050fa617a3200c8cf29271dc8bbdf8ae0ee39bf6f1672fd291e9b9e1b7 WatchSource:0}: Error finding container da0ce5050fa617a3200c8cf29271dc8bbdf8ae0ee39bf6f1672fd291e9b9e1b7: Status 404 returned error can't find the container with id da0ce5050fa617a3200c8cf29271dc8bbdf8ae0ee39bf6f1672fd291e9b9e1b7
Jul 06 22:30:11.705730 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:30:11.705656    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-z2tm5" event={"ID":"8fe2c7e3-c0da-4480-90f8-d17046019fdd","Type":"ContainerStarted","Data":"da0ce5050fa617a3200c8cf29271dc8bbdf8ae0ee39bf6f1672fd291e9b9e1b7"}
Jul 06 22:30:13.288436 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:30:13.288400    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-chgfg"]
Jul 06 22:30:13.295726 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:30:13.295704    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-chgfg"
Jul 06 22:30:13.298340 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:30:13.298314    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Jul 06 22:30:13.298896 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:30:13.298872    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-ksp9k\""
Jul 06 22:30:13.300635 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:30:13.300608    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-mmb27"]
Jul 06 22:30:13.306235 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:30:13.306162    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-chgfg"]
Jul 06 22:30:13.306320 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:30:13.306275    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mmb27"
Jul 06 22:30:13.312803 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:30:13.312763    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-gqjgx\""
Jul 06 22:30:13.313025 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:30:13.312904    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Jul 06 22:30:13.318447 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:30:13.318428    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-mmb27"]
Jul 06 22:30:13.396164 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:30:13.396126    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xkmlr\" (UniqueName: \"kubernetes.io/projected/42e3efe9-17e9-4b0f-8ef6-f3b7922ae8a5-kube-api-access-xkmlr\") pod \"kserve-controller-manager-d6f4b65b8-chgfg\" (UID: \"42e3efe9-17e9-4b0f-8ef6-f3b7922ae8a5\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-chgfg"
Jul 06 22:30:13.396357 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:30:13.396196    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/42e3efe9-17e9-4b0f-8ef6-f3b7922ae8a5-cert\") pod \"kserve-controller-manager-d6f4b65b8-chgfg\" (UID: \"42e3efe9-17e9-4b0f-8ef6-f3b7922ae8a5\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-chgfg"
Jul 06 22:30:13.396357 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:30:13.396227    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b7051b49-7c43-4cab-a533-2e2c9b64fddf-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-mmb27\" (UID: \"b7051b49-7c43-4cab-a533-2e2c9b64fddf\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mmb27"
Jul 06 22:30:13.396357 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:30:13.396265    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fpkbh\" (UniqueName: \"kubernetes.io/projected/b7051b49-7c43-4cab-a533-2e2c9b64fddf-kube-api-access-fpkbh\") pod \"llmisvc-controller-manager-777c6bd7d5-mmb27\" (UID: \"b7051b49-7c43-4cab-a533-2e2c9b64fddf\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mmb27"
Jul 06 22:30:13.497524 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:30:13.497487    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xkmlr\" (UniqueName: \"kubernetes.io/projected/42e3efe9-17e9-4b0f-8ef6-f3b7922ae8a5-kube-api-access-xkmlr\") pod \"kserve-controller-manager-d6f4b65b8-chgfg\" (UID: \"42e3efe9-17e9-4b0f-8ef6-f3b7922ae8a5\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-chgfg"
Jul 06 22:30:13.497685 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:30:13.497534    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/42e3efe9-17e9-4b0f-8ef6-f3b7922ae8a5-cert\") pod \"kserve-controller-manager-d6f4b65b8-chgfg\" (UID: \"42e3efe9-17e9-4b0f-8ef6-f3b7922ae8a5\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-chgfg"
Jul 06 22:30:13.497685 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:30:13.497563    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b7051b49-7c43-4cab-a533-2e2c9b64fddf-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-mmb27\" (UID: \"b7051b49-7c43-4cab-a533-2e2c9b64fddf\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mmb27"
Jul 06 22:30:13.497685 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:30:13.497600    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fpkbh\" (UniqueName: \"kubernetes.io/projected/b7051b49-7c43-4cab-a533-2e2c9b64fddf-kube-api-access-fpkbh\") pod \"llmisvc-controller-manager-777c6bd7d5-mmb27\" (UID: \"b7051b49-7c43-4cab-a533-2e2c9b64fddf\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mmb27"
Jul 06 22:30:13.500464 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:30:13.500436    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b7051b49-7c43-4cab-a533-2e2c9b64fddf-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-mmb27\" (UID: \"b7051b49-7c43-4cab-a533-2e2c9b64fddf\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mmb27"
Jul 06 22:30:13.500464 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:30:13.500453    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/42e3efe9-17e9-4b0f-8ef6-f3b7922ae8a5-cert\") pod \"kserve-controller-manager-d6f4b65b8-chgfg\" (UID: \"42e3efe9-17e9-4b0f-8ef6-f3b7922ae8a5\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-chgfg"
Jul 06 22:30:13.523568 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:30:13.523532    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fpkbh\" (UniqueName: \"kubernetes.io/projected/b7051b49-7c43-4cab-a533-2e2c9b64fddf-kube-api-access-fpkbh\") pod \"llmisvc-controller-manager-777c6bd7d5-mmb27\" (UID: \"b7051b49-7c43-4cab-a533-2e2c9b64fddf\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mmb27"
Jul 06 22:30:13.524164 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:30:13.524141    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xkmlr\" (UniqueName: \"kubernetes.io/projected/42e3efe9-17e9-4b0f-8ef6-f3b7922ae8a5-kube-api-access-xkmlr\") pod \"kserve-controller-manager-d6f4b65b8-chgfg\" (UID: \"42e3efe9-17e9-4b0f-8ef6-f3b7922ae8a5\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-chgfg"
Jul 06 22:30:13.592052 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:30:13.592015    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 06 22:30:13.592194 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:30:13.592100    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 06 22:30:13.610469 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:30:13.610448    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-chgfg"
Jul 06 22:30:13.622317 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:30:13.622297    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mmb27"
Jul 06 22:30:13.719517 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:30:13.718429    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-z2tm5" event={"ID":"8fe2c7e3-c0da-4480-90f8-d17046019fdd","Type":"ContainerStarted","Data":"6b1e9731f1efe38b7ff794ca986702b6380b8a60e7597dc9ed41dc0c550538ea"}
Jul 06 22:30:13.719517 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:30:13.719307    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-z2tm5"
Jul 06 22:30:13.746354 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:30:13.746296    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-z2tm5" podStartSLOduration=1.473011803 podStartE2EDuration="3.746272791s" totalImagesPullingTime="2.273260988s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:30:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:30:13.744083602 +0000 UTC m=+379.169130144" watchObservedRunningTime="2026-07-06 22:30:13.746272791 +0000 UTC m=+379.171319334"
Jul 06 22:30:13.773415 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:30:13.773385    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-chgfg"]
Jul 06 22:30:13.775501 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:30:13.775472    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42e3efe9_17e9_4b0f_8ef6_f3b7922ae8a5.slice/crio-3487ec643133ea34edbd2c6232a3e69863d978c1e6a9c71a9639d15a618d5539 WatchSource:0}: Error finding container 3487ec643133ea34edbd2c6232a3e69863d978c1e6a9c71a9639d15a618d5539: Status 404 returned error can't find the container with id 3487ec643133ea34edbd2c6232a3e69863d978c1e6a9c71a9639d15a618d5539
Jul 06 22:30:13.794523 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:30:13.794497    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-mmb27"]
Jul 06 22:30:13.796708 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:30:13.796682    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podb7051b49_7c43_4cab_a533_2e2c9b64fddf.slice/crio-98b04998be1149826df6b1dbd9aaa3dfe39c92688804db37932b0c68f7bb7e1c WatchSource:0}: Error finding container 98b04998be1149826df6b1dbd9aaa3dfe39c92688804db37932b0c68f7bb7e1c: Status 404 returned error can't find the container with id 98b04998be1149826df6b1dbd9aaa3dfe39c92688804db37932b0c68f7bb7e1c
Jul 06 22:30:14.726656 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:30:14.726582    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-chgfg" event={"ID":"42e3efe9-17e9-4b0f-8ef6-f3b7922ae8a5","Type":"ContainerStarted","Data":"3487ec643133ea34edbd2c6232a3e69863d978c1e6a9c71a9639d15a618d5539"}
Jul 06 22:30:14.729147 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:30:14.729080    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mmb27" event={"ID":"b7051b49-7c43-4cab-a533-2e2c9b64fddf","Type":"ContainerStarted","Data":"98b04998be1149826df6b1dbd9aaa3dfe39c92688804db37932b0c68f7bb7e1c"}
Jul 06 22:30:14.732314 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:30:14.732290    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-z2tm5"
Jul 06 22:30:17.690630 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:30:17.690600    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-r5zbk"
Jul 06 22:30:17.757461 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:30:17.757423    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-chgfg" event={"ID":"42e3efe9-17e9-4b0f-8ef6-f3b7922ae8a5","Type":"ContainerStarted","Data":"231ae7718127086e50170b3bd189e9062baf402c5793502051be797c4e5039ce"}
Jul 06 22:30:17.757657 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:30:17.757574    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-chgfg"
Jul 06 22:30:17.759462 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:30:17.759436    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mmb27" event={"ID":"b7051b49-7c43-4cab-a533-2e2c9b64fddf","Type":"ContainerStarted","Data":"7c135f956225a26e29966b9e4df4fb40799f229c0349ed309a4bb80e8ae65a1a"}
Jul 06 22:30:17.759594 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:30:17.759568    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mmb27"
Jul 06 22:30:17.776249 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:30:17.776200    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-chgfg" podStartSLOduration=1.886002059 podStartE2EDuration="4.776184553s" totalImagesPullingTime="2.890182494s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:30:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:30:17.774342071 +0000 UTC m=+383.199388616" watchObservedRunningTime="2026-07-06 22:30:17.776184553 +0000 UTC m=+383.201231086"
Jul 06 22:30:17.791032 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:30:17.790964    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mmb27" podStartSLOduration=1.918223542 podStartE2EDuration="4.790948161s" totalImagesPullingTime="2.872724619s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:30:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:30:17.790446476 +0000 UTC m=+383.215493017" watchObservedRunningTime="2026-07-06 22:30:17.790948161 +0000 UTC m=+383.215994703"
Jul 06 22:30:48.766024 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:30:48.765993    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mmb27"
Jul 06 22:30:48.769018 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:30:48.769000    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-chgfg"
Jul 06 22:31:33.905157 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:31:33.905076    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-bwrkv"]
Jul 06 22:31:33.907452 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:31:33.907436    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-bwrkv"
Jul 06 22:31:33.910139 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:31:33.910115    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 06 22:31:33.911210 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:31:33.911190    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-t2rb5\""
Jul 06 22:31:33.911311 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:31:33.911239    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 06 22:31:33.924135 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:31:33.924108    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-bwrkv"]
Jul 06 22:31:33.982779 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:31:33.982754    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pcnsg\" (UniqueName: \"kubernetes.io/projected/3bcdb607-d473-4695-b8d2-fc3431446f95-kube-api-access-pcnsg\") pod \"authorino-operator-75cc8d5cb7-bwrkv\" (UID: \"3bcdb607-d473-4695-b8d2-fc3431446f95\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-bwrkv"
Jul 06 22:31:34.083733 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:31:34.083696    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pcnsg\" (UniqueName: \"kubernetes.io/projected/3bcdb607-d473-4695-b8d2-fc3431446f95-kube-api-access-pcnsg\") pod \"authorino-operator-75cc8d5cb7-bwrkv\" (UID: \"3bcdb607-d473-4695-b8d2-fc3431446f95\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-bwrkv"
Jul 06 22:31:34.096071 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:31:34.096047    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pcnsg\" (UniqueName: \"kubernetes.io/projected/3bcdb607-d473-4695-b8d2-fc3431446f95-kube-api-access-pcnsg\") pod \"authorino-operator-75cc8d5cb7-bwrkv\" (UID: \"3bcdb607-d473-4695-b8d2-fc3431446f95\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-bwrkv"
Jul 06 22:31:34.217789 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:31:34.217758    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-bwrkv"
Jul 06 22:31:34.380030 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:31:34.379991    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-bwrkv"]
Jul 06 22:31:34.388829 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:31:34.388802    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3bcdb607_d473_4695_b8d2_fc3431446f95.slice/crio-bf64400af95d6a07faa353c098e4d932cdafdb11cb600234fe90f9949b6a61c5 WatchSource:0}: Error finding container bf64400af95d6a07faa353c098e4d932cdafdb11cb600234fe90f9949b6a61c5: Status 404 returned error can't find the container with id bf64400af95d6a07faa353c098e4d932cdafdb11cb600234fe90f9949b6a61c5
Jul 06 22:31:35.082390 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:31:35.082355    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-bwrkv" event={"ID":"3bcdb607-d473-4695-b8d2-fc3431446f95","Type":"ContainerStarted","Data":"bf64400af95d6a07faa353c098e4d932cdafdb11cb600234fe90f9949b6a61c5"}
Jul 06 22:31:37.089405 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:31:37.089369    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-bwrkv" event={"ID":"3bcdb607-d473-4695-b8d2-fc3431446f95","Type":"ContainerStarted","Data":"7800a97201b8e0910061cd32b79bd832bcb331ddb8f8eb614a915e09fdc92894"}
Jul 06 22:31:37.089844 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:31:37.089504    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-bwrkv"
Jul 06 22:31:37.153976 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:31:37.153924    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-bwrkv" podStartSLOduration=2.079879193 podStartE2EDuration="4.153910479s" totalImagesPullingTime="2.074031286s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:31:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:31:37.151359661 +0000 UTC m=+462.576406214" watchObservedRunningTime="2026-07-06 22:31:37.153910479 +0000 UTC m=+462.578957020"
Jul 06 22:31:42.224951 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:31:42.224914    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4tj5n"]
Jul 06 22:31:42.227819 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:31:42.227797    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4tj5n"
Jul 06 22:31:42.230777 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:31:42.230756    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-bzlzn\""
Jul 06 22:31:42.241223 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:31:42.241199    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4tj5n"]
Jul 06 22:31:42.362053 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:31:42.362024    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f47266ea-8d00-4be4-ae8e-40a69cf50013-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-4tj5n\" (UID: \"f47266ea-8d00-4be4-ae8e-40a69cf50013\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4tj5n"
Jul 06 22:31:42.362221 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:31:42.362058    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rrdn9\" (UniqueName: \"kubernetes.io/projected/f47266ea-8d00-4be4-ae8e-40a69cf50013-kube-api-access-rrdn9\") pod \"kuadrant-operator-controller-manager-8684c9ff49-4tj5n\" (UID: \"f47266ea-8d00-4be4-ae8e-40a69cf50013\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4tj5n"
Jul 06 22:31:42.462690 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:31:42.462654    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f47266ea-8d00-4be4-ae8e-40a69cf50013-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-4tj5n\" (UID: \"f47266ea-8d00-4be4-ae8e-40a69cf50013\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4tj5n"
Jul 06 22:31:42.462690 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:31:42.462691    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rrdn9\" (UniqueName: \"kubernetes.io/projected/f47266ea-8d00-4be4-ae8e-40a69cf50013-kube-api-access-rrdn9\") pod \"kuadrant-operator-controller-manager-8684c9ff49-4tj5n\" (UID: \"f47266ea-8d00-4be4-ae8e-40a69cf50013\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4tj5n"
Jul 06 22:31:42.463116 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:31:42.463094    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f47266ea-8d00-4be4-ae8e-40a69cf50013-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-4tj5n\" (UID: \"f47266ea-8d00-4be4-ae8e-40a69cf50013\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4tj5n"
Jul 06 22:31:42.498050 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:31:42.497984    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rrdn9\" (UniqueName: \"kubernetes.io/projected/f47266ea-8d00-4be4-ae8e-40a69cf50013-kube-api-access-rrdn9\") pod \"kuadrant-operator-controller-manager-8684c9ff49-4tj5n\" (UID: \"f47266ea-8d00-4be4-ae8e-40a69cf50013\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4tj5n"
Jul 06 22:31:42.539022 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:31:42.538987    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4tj5n"
Jul 06 22:31:42.673489 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:31:42.673401    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4tj5n"]
Jul 06 22:31:42.675836 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:31:42.675809    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf47266ea_8d00_4be4_ae8e_40a69cf50013.slice/crio-b23237ea3d4c66b0356b8472d668f201b1da462d80c23bd1e195db83e5871687 WatchSource:0}: Error finding container b23237ea3d4c66b0356b8472d668f201b1da462d80c23bd1e195db83e5871687: Status 404 returned error can't find the container with id b23237ea3d4c66b0356b8472d668f201b1da462d80c23bd1e195db83e5871687
Jul 06 22:31:43.114232 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:31:43.114200    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4tj5n" event={"ID":"f47266ea-8d00-4be4-ae8e-40a69cf50013","Type":"ContainerStarted","Data":"b23237ea3d4c66b0356b8472d668f201b1da462d80c23bd1e195db83e5871687"}
Jul 06 22:31:48.095755 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:31:48.095724    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-bwrkv"
Jul 06 22:31:48.140399 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:31:48.140367    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4tj5n" event={"ID":"f47266ea-8d00-4be4-ae8e-40a69cf50013","Type":"ContainerStarted","Data":"da81507b112bbff1667e5b420002d998acbb0d2b9edc82554d696af5c85da42e"}
Jul 06 22:31:48.140566 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:31:48.140450    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4tj5n"
Jul 06 22:31:48.161994 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:31:48.161937    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4tj5n" podStartSLOduration=1.245762071 podStartE2EDuration="6.161918733s" totalImagesPullingTime="4.916156662s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:31:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:31:48.159247671 +0000 UTC m=+473.584294212" watchObservedRunningTime="2026-07-06 22:31:48.161918733 +0000 UTC m=+473.586965277"
Jul 06 22:31:59.147193 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:31:59.147143    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4tj5n"
Jul 06 22:32:00.861966 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:32:00.861930    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4tj5n"]
Jul 06 22:32:00.862396 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:32:00.862192    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4tj5n" podUID="f47266ea-8d00-4be4-ae8e-40a69cf50013" containerName="manager" containerID="cri-o://da81507b112bbff1667e5b420002d998acbb0d2b9edc82554d696af5c85da42e" gracePeriod=2
Jul 06 22:32:00.872204 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:32:00.872159    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4tj5n"]
Jul 06 22:32:00.873975 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:32:00.873941    2563 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"f47266ea-8d00-4be4-ae8e-40a69cf50013\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-8684c9ff49-4tj5n\": pods \"kuadrant-operator-controller-manager-8684c9ff49-4tj5n\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4tj5n"
Jul 06 22:32:00.896297 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:32:00.896265    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wmzp8"]
Jul 06 22:32:00.896790 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:32:00.896770    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f47266ea-8d00-4be4-ae8e-40a69cf50013" containerName="manager"
Jul 06 22:32:00.896853 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:32:00.896795    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="f47266ea-8d00-4be4-ae8e-40a69cf50013" containerName="manager"
Jul 06 22:32:00.896888 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:32:00.896879    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="f47266ea-8d00-4be4-ae8e-40a69cf50013" containerName="manager"
Jul 06 22:32:00.900209 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:32:00.900153    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wmzp8"
Jul 06 22:32:00.902513 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:32:00.902483    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-4tj5n\" is forbidden: User \"system:node:ip-10-0-133-105.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-105.ec2.internal' and this object" podUID="f47266ea-8d00-4be4-ae8e-40a69cf50013" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4tj5n"
Jul 06 22:32:00.917130 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:32:00.917103    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wmzp8"]
Jul 06 22:32:01.021350 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:32:01.021316    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b4mdd\" (UniqueName: \"kubernetes.io/projected/2b6b918a-ad7a-4d38-b673-82a6cfe31519-kube-api-access-b4mdd\") pod \"kuadrant-operator-controller-manager-8684c9ff49-wmzp8\" (UID: \"2b6b918a-ad7a-4d38-b673-82a6cfe31519\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wmzp8"
Jul 06 22:32:01.021511 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:32:01.021365    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2b6b918a-ad7a-4d38-b673-82a6cfe31519-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-wmzp8\" (UID: \"2b6b918a-ad7a-4d38-b673-82a6cfe31519\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wmzp8"
Jul 06 22:32:01.101578 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:32:01.101555    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4tj5n"
Jul 06 22:32:01.122887 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:32:01.122829    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2b6b918a-ad7a-4d38-b673-82a6cfe31519-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-wmzp8\" (UID: \"2b6b918a-ad7a-4d38-b673-82a6cfe31519\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wmzp8"
Jul 06 22:32:01.122977 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:32:01.122935    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b4mdd\" (UniqueName: \"kubernetes.io/projected/2b6b918a-ad7a-4d38-b673-82a6cfe31519-kube-api-access-b4mdd\") pod \"kuadrant-operator-controller-manager-8684c9ff49-wmzp8\" (UID: \"2b6b918a-ad7a-4d38-b673-82a6cfe31519\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wmzp8"
Jul 06 22:32:01.123263 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:32:01.123244    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2b6b918a-ad7a-4d38-b673-82a6cfe31519-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-wmzp8\" (UID: \"2b6b918a-ad7a-4d38-b673-82a6cfe31519\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wmzp8"
Jul 06 22:32:01.138597 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:32:01.138568    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b4mdd\" (UniqueName: \"kubernetes.io/projected/2b6b918a-ad7a-4d38-b673-82a6cfe31519-kube-api-access-b4mdd\") pod \"kuadrant-operator-controller-manager-8684c9ff49-wmzp8\" (UID: \"2b6b918a-ad7a-4d38-b673-82a6cfe31519\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wmzp8"
Jul 06 22:32:01.197639 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:32:01.197612    2563 generic.go:356] "Generic (PLEG): container finished" podID="f47266ea-8d00-4be4-ae8e-40a69cf50013" containerID="da81507b112bbff1667e5b420002d998acbb0d2b9edc82554d696af5c85da42e" exitCode=0
Jul 06 22:32:01.197774 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:32:01.197659    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4tj5n"
Jul 06 22:32:01.197774 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:32:01.197719    2563 scope.go:117] "RemoveContainer" containerID="da81507b112bbff1667e5b420002d998acbb0d2b9edc82554d696af5c85da42e"
Jul 06 22:32:01.205838 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:32:01.205822    2563 scope.go:117] "RemoveContainer" containerID="da81507b112bbff1667e5b420002d998acbb0d2b9edc82554d696af5c85da42e"
Jul 06 22:32:01.206071 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:32:01.206055    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"da81507b112bbff1667e5b420002d998acbb0d2b9edc82554d696af5c85da42e\": container with ID starting with da81507b112bbff1667e5b420002d998acbb0d2b9edc82554d696af5c85da42e not found: ID does not exist" containerID="da81507b112bbff1667e5b420002d998acbb0d2b9edc82554d696af5c85da42e"
Jul 06 22:32:01.206120 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:32:01.206079    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"da81507b112bbff1667e5b420002d998acbb0d2b9edc82554d696af5c85da42e"} err="failed to get container status \"da81507b112bbff1667e5b420002d998acbb0d2b9edc82554d696af5c85da42e\": rpc error: code = NotFound desc = could not find container \"da81507b112bbff1667e5b420002d998acbb0d2b9edc82554d696af5c85da42e\": container with ID starting with da81507b112bbff1667e5b420002d998acbb0d2b9edc82554d696af5c85da42e not found: ID does not exist"
Jul 06 22:32:01.223864 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:32:01.223840    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f47266ea-8d00-4be4-ae8e-40a69cf50013-extensions-socket-volume\") pod \"f47266ea-8d00-4be4-ae8e-40a69cf50013\" (UID: \"f47266ea-8d00-4be4-ae8e-40a69cf50013\") "
Jul 06 22:32:01.223964 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:32:01.223887    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rrdn9\" (UniqueName: \"kubernetes.io/projected/f47266ea-8d00-4be4-ae8e-40a69cf50013-kube-api-access-rrdn9\") pod \"f47266ea-8d00-4be4-ae8e-40a69cf50013\" (UID: \"f47266ea-8d00-4be4-ae8e-40a69cf50013\") "
Jul 06 22:32:01.224226 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:32:01.224204    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f47266ea-8d00-4be4-ae8e-40a69cf50013-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "f47266ea-8d00-4be4-ae8e-40a69cf50013" (UID: "f47266ea-8d00-4be4-ae8e-40a69cf50013"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 22:32:01.226042 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:32:01.226017    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f47266ea-8d00-4be4-ae8e-40a69cf50013-kube-api-access-rrdn9" (OuterVolumeSpecName: "kube-api-access-rrdn9") pod "f47266ea-8d00-4be4-ae8e-40a69cf50013" (UID: "f47266ea-8d00-4be4-ae8e-40a69cf50013"). InnerVolumeSpecName "kube-api-access-rrdn9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 22:32:01.230272 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:32:01.230258    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wmzp8"
Jul 06 22:32:01.324933 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:32:01.324908    2563 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f47266ea-8d00-4be4-ae8e-40a69cf50013-extensions-socket-volume\") on node \"ip-10-0-133-105.ec2.internal\" DevicePath \"\""
Jul 06 22:32:01.324933 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:32:01.324933    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rrdn9\" (UniqueName: \"kubernetes.io/projected/f47266ea-8d00-4be4-ae8e-40a69cf50013-kube-api-access-rrdn9\") on node \"ip-10-0-133-105.ec2.internal\" DevicePath \"\""
Jul 06 22:32:01.373158 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:32:01.373104    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wmzp8"]
Jul 06 22:32:01.376076 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:32:01.376049    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b6b918a_ad7a_4d38_b673_82a6cfe31519.slice/crio-1bbab976d12b783e77727ef85618349a802f0ba1fbc553d4cf3e3204daa2c059 WatchSource:0}: Error finding container 1bbab976d12b783e77727ef85618349a802f0ba1fbc553d4cf3e3204daa2c059: Status 404 returned error can't find the container with id 1bbab976d12b783e77727ef85618349a802f0ba1fbc553d4cf3e3204daa2c059
Jul 06 22:32:02.205599 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:32:02.205555    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wmzp8" event={"ID":"2b6b918a-ad7a-4d38-b673-82a6cfe31519","Type":"ContainerStarted","Data":"c8bf8bab32794a1aeea2ad319afb7ccc08654e62065c5438f85f4826f359e0d0"}
Jul 06 22:32:02.205985 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:32:02.205615    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wmzp8" event={"ID":"2b6b918a-ad7a-4d38-b673-82a6cfe31519","Type":"ContainerStarted","Data":"1bbab976d12b783e77727ef85618349a802f0ba1fbc553d4cf3e3204daa2c059"}
Jul 06 22:32:02.205985 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:32:02.205704    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wmzp8"
Jul 06 22:32:02.238416 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:32:02.238346    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wmzp8" podStartSLOduration=2.238325917 podStartE2EDuration="2.238325917s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:32:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:32:02.23677189 +0000 UTC m=+487.661818432" watchObservedRunningTime="2026-07-06 22:32:02.238325917 +0000 UTC m=+487.663372458"
Jul 06 22:32:03.080455 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:32:03.080423    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f47266ea-8d00-4be4-ae8e-40a69cf50013" path="/var/lib/kubelet/pods/f47266ea-8d00-4be4-ae8e-40a69cf50013/volumes"
Jul 06 22:32:13.212255 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:32:13.212216    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wmzp8"
Jul 06 22:32:28.776377 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:32:28.776346    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wmzp8"]
Jul 06 22:32:28.776794 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:32:28.776533    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wmzp8" podUID="2b6b918a-ad7a-4d38-b673-82a6cfe31519" containerName="manager" containerID="cri-o://c8bf8bab32794a1aeea2ad319afb7ccc08654e62065c5438f85f4826f359e0d0" gracePeriod=10
Jul 06 22:32:29.028849 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:32:29.028796    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wmzp8"
Jul 06 22:32:29.062299 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:32:29.062251    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-h559w"]
Jul 06 22:32:29.062584 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:32:29.062572    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2b6b918a-ad7a-4d38-b673-82a6cfe31519" containerName="manager"
Jul 06 22:32:29.062627 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:32:29.062586    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="2b6b918a-ad7a-4d38-b673-82a6cfe31519" containerName="manager"
Jul 06 22:32:29.062660 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:32:29.062643    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="2b6b918a-ad7a-4d38-b673-82a6cfe31519" containerName="manager"
Jul 06 22:32:29.065083 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:32:29.065064    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-h559w"
Jul 06 22:32:29.082512 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:32:29.082492    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-h559w"]
Jul 06 22:32:29.160000 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:32:29.156604    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2b6b918a-ad7a-4d38-b673-82a6cfe31519-extensions-socket-volume\") pod \"2b6b918a-ad7a-4d38-b673-82a6cfe31519\" (UID: \"2b6b918a-ad7a-4d38-b673-82a6cfe31519\") "
Jul 06 22:32:29.160000 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:32:29.156673    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b4mdd\" (UniqueName: \"kubernetes.io/projected/2b6b918a-ad7a-4d38-b673-82a6cfe31519-kube-api-access-b4mdd\") pod \"2b6b918a-ad7a-4d38-b673-82a6cfe31519\" (UID: \"2b6b918a-ad7a-4d38-b673-82a6cfe31519\") "
Jul 06 22:32:29.160000 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:32:29.157088    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-88xqb\" (UniqueName: \"kubernetes.io/projected/782bd269-b2e7-4411-b3d9-39bf3466cfb2-kube-api-access-88xqb\") pod \"kuadrant-operator-controller-manager-86878c769f-h559w\" (UID: \"782bd269-b2e7-4411-b3d9-39bf3466cfb2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-h559w"
Jul 06 22:32:29.160000 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:32:29.157145    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/782bd269-b2e7-4411-b3d9-39bf3466cfb2-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-h559w\" (UID: \"782bd269-b2e7-4411-b3d9-39bf3466cfb2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-h559w"
Jul 06 22:32:29.160000 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:32:29.157640    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b6b918a-ad7a-4d38-b673-82a6cfe31519-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "2b6b918a-ad7a-4d38-b673-82a6cfe31519" (UID: "2b6b918a-ad7a-4d38-b673-82a6cfe31519"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 22:32:29.161999 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:32:29.160394    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2b6b918a-ad7a-4d38-b673-82a6cfe31519-kube-api-access-b4mdd" (OuterVolumeSpecName: "kube-api-access-b4mdd") pod "2b6b918a-ad7a-4d38-b673-82a6cfe31519" (UID: "2b6b918a-ad7a-4d38-b673-82a6cfe31519"). InnerVolumeSpecName "kube-api-access-b4mdd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 22:32:29.257593 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:32:29.257560    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-88xqb\" (UniqueName: \"kubernetes.io/projected/782bd269-b2e7-4411-b3d9-39bf3466cfb2-kube-api-access-88xqb\") pod \"kuadrant-operator-controller-manager-86878c769f-h559w\" (UID: \"782bd269-b2e7-4411-b3d9-39bf3466cfb2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-h559w"
Jul 06 22:32:29.257773 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:32:29.257603    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/782bd269-b2e7-4411-b3d9-39bf3466cfb2-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-h559w\" (UID: \"782bd269-b2e7-4411-b3d9-39bf3466cfb2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-h559w"
Jul 06 22:32:29.257773 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:32:29.257706    2563 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2b6b918a-ad7a-4d38-b673-82a6cfe31519-extensions-socket-volume\") on node \"ip-10-0-133-105.ec2.internal\" DevicePath \"\""
Jul 06 22:32:29.257773 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:32:29.257722    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b4mdd\" (UniqueName: \"kubernetes.io/projected/2b6b918a-ad7a-4d38-b673-82a6cfe31519-kube-api-access-b4mdd\") on node \"ip-10-0-133-105.ec2.internal\" DevicePath \"\""
Jul 06 22:32:29.258056 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:32:29.258036    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/782bd269-b2e7-4411-b3d9-39bf3466cfb2-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-h559w\" (UID: \"782bd269-b2e7-4411-b3d9-39bf3466cfb2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-h559w"
Jul 06 22:32:29.268303 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:32:29.268282    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-88xqb\" (UniqueName: \"kubernetes.io/projected/782bd269-b2e7-4411-b3d9-39bf3466cfb2-kube-api-access-88xqb\") pod \"kuadrant-operator-controller-manager-86878c769f-h559w\" (UID: \"782bd269-b2e7-4411-b3d9-39bf3466cfb2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-h559w"
Jul 06 22:32:29.323824 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:32:29.323728    2563 generic.go:356] "Generic (PLEG): container finished" podID="2b6b918a-ad7a-4d38-b673-82a6cfe31519" containerID="c8bf8bab32794a1aeea2ad319afb7ccc08654e62065c5438f85f4826f359e0d0" exitCode=0
Jul 06 22:32:29.323824 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:32:29.323800    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wmzp8"
Jul 06 22:32:29.324024 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:32:29.323812    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wmzp8" event={"ID":"2b6b918a-ad7a-4d38-b673-82a6cfe31519","Type":"ContainerDied","Data":"c8bf8bab32794a1aeea2ad319afb7ccc08654e62065c5438f85f4826f359e0d0"}
Jul 06 22:32:29.324024 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:32:29.323893    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wmzp8" event={"ID":"2b6b918a-ad7a-4d38-b673-82a6cfe31519","Type":"ContainerDied","Data":"1bbab976d12b783e77727ef85618349a802f0ba1fbc553d4cf3e3204daa2c059"}
Jul 06 22:32:29.324024 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:32:29.323915    2563 scope.go:117] "RemoveContainer" containerID="c8bf8bab32794a1aeea2ad319afb7ccc08654e62065c5438f85f4826f359e0d0"
Jul 06 22:32:29.332667 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:32:29.332651    2563 scope.go:117] "RemoveContainer" containerID="c8bf8bab32794a1aeea2ad319afb7ccc08654e62065c5438f85f4826f359e0d0"
Jul 06 22:32:29.332905 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:32:29.332886    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c8bf8bab32794a1aeea2ad319afb7ccc08654e62065c5438f85f4826f359e0d0\": container with ID starting with c8bf8bab32794a1aeea2ad319afb7ccc08654e62065c5438f85f4826f359e0d0 not found: ID does not exist" containerID="c8bf8bab32794a1aeea2ad319afb7ccc08654e62065c5438f85f4826f359e0d0"
Jul 06 22:32:29.332947 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:32:29.332914    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c8bf8bab32794a1aeea2ad319afb7ccc08654e62065c5438f85f4826f359e0d0"} err="failed to get container status \"c8bf8bab32794a1aeea2ad319afb7ccc08654e62065c5438f85f4826f359e0d0\": rpc error: code = NotFound desc = could not find container \"c8bf8bab32794a1aeea2ad319afb7ccc08654e62065c5438f85f4826f359e0d0\": container with ID starting with c8bf8bab32794a1aeea2ad319afb7ccc08654e62065c5438f85f4826f359e0d0 not found: ID does not exist"
Jul 06 22:32:29.344729 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:32:29.344708    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wmzp8"]
Jul 06 22:32:29.350651 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:32:29.350632    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-wmzp8"]
Jul 06 22:32:29.375263 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:32:29.375236    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-h559w"
Jul 06 22:32:29.517919 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:32:29.517844    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-h559w"]
Jul 06 22:32:29.520443 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:32:29.520405    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod782bd269_b2e7_4411_b3d9_39bf3466cfb2.slice/crio-73cdb3057f2fd7e6b3d23c2bb48cf2f6c9342090ead6b2cd5113af24e2d21bbe WatchSource:0}: Error finding container 73cdb3057f2fd7e6b3d23c2bb48cf2f6c9342090ead6b2cd5113af24e2d21bbe: Status 404 returned error can't find the container with id 73cdb3057f2fd7e6b3d23c2bb48cf2f6c9342090ead6b2cd5113af24e2d21bbe
Jul 06 22:32:30.329973 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:32:30.329936    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-h559w" event={"ID":"782bd269-b2e7-4411-b3d9-39bf3466cfb2","Type":"ContainerStarted","Data":"af0d5de136b279b2c867dc5dd3785005acdde4e970f29497a1ae8976ec2f286b"}
Jul 06 22:32:30.330441 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:32:30.330005    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-h559w"
Jul 06 22:32:30.330441 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:32:30.330031    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-h559w" event={"ID":"782bd269-b2e7-4411-b3d9-39bf3466cfb2","Type":"ContainerStarted","Data":"73cdb3057f2fd7e6b3d23c2bb48cf2f6c9342090ead6b2cd5113af24e2d21bbe"}
Jul 06 22:32:30.352598 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:32:30.352547    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-h559w" podStartSLOduration=1.352530755 podStartE2EDuration="1.352530755s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:32:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:32:30.351581201 +0000 UTC m=+515.776627743" watchObservedRunningTime="2026-07-06 22:32:30.352530755 +0000 UTC m=+515.777577296"
Jul 06 22:32:31.079722 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:32:31.079691    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2b6b918a-ad7a-4d38-b673-82a6cfe31519" path="/var/lib/kubelet/pods/2b6b918a-ad7a-4d38-b673-82a6cfe31519/volumes"
Jul 06 22:32:41.337820 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:32:41.337788    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-h559w"
Jul 06 22:32:50.289470 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:32:50.289392    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9q89m"]
Jul 06 22:32:50.292583 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:32:50.292561    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-9q89m"
Jul 06 22:32:50.295229 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:32:50.295204    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 06 22:32:50.295229 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:32:50.295206    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-n9m9x\""
Jul 06 22:32:50.302829 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:32:50.302803    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9q89m"]
Jul 06 22:32:50.314673 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:32:50.314647    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9q89m"]
Jul 06 22:32:50.440324 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:32:50.440295    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/53a67b97-ec4a-4505-8fd7-fc6bec48335a-config-file\") pod \"limitador-limitador-54d6d988d5-9q89m\" (UID: \"53a67b97-ec4a-4505-8fd7-fc6bec48335a\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-9q89m"
Jul 06 22:32:50.440487 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:32:50.440333    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vhjdt\" (UniqueName: \"kubernetes.io/projected/53a67b97-ec4a-4505-8fd7-fc6bec48335a-kube-api-access-vhjdt\") pod \"limitador-limitador-54d6d988d5-9q89m\" (UID: \"53a67b97-ec4a-4505-8fd7-fc6bec48335a\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-9q89m"
Jul 06 22:32:50.541355 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:32:50.541262    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/53a67b97-ec4a-4505-8fd7-fc6bec48335a-config-file\") pod \"limitador-limitador-54d6d988d5-9q89m\" (UID: \"53a67b97-ec4a-4505-8fd7-fc6bec48335a\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-9q89m"
Jul 06 22:32:50.541355 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:32:50.541323    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vhjdt\" (UniqueName: \"kubernetes.io/projected/53a67b97-ec4a-4505-8fd7-fc6bec48335a-kube-api-access-vhjdt\") pod \"limitador-limitador-54d6d988d5-9q89m\" (UID: \"53a67b97-ec4a-4505-8fd7-fc6bec48335a\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-9q89m"
Jul 06 22:32:50.541961 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:32:50.541942    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/53a67b97-ec4a-4505-8fd7-fc6bec48335a-config-file\") pod \"limitador-limitador-54d6d988d5-9q89m\" (UID: \"53a67b97-ec4a-4505-8fd7-fc6bec48335a\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-9q89m"
Jul 06 22:32:50.549863 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:32:50.549840    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vhjdt\" (UniqueName: \"kubernetes.io/projected/53a67b97-ec4a-4505-8fd7-fc6bec48335a-kube-api-access-vhjdt\") pod \"limitador-limitador-54d6d988d5-9q89m\" (UID: \"53a67b97-ec4a-4505-8fd7-fc6bec48335a\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-9q89m"
Jul 06 22:32:50.603988 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:32:50.603964    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-9q89m"
Jul 06 22:32:50.741766 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:32:50.741738    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9q89m"]
Jul 06 22:32:50.745400 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:32:50.745348    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53a67b97_ec4a_4505_8fd7_fc6bec48335a.slice/crio-d45de9515f8975d862bb71617e3dbb0fcb8b2fff1696a2bc04fecfbf60b73696 WatchSource:0}: Error finding container d45de9515f8975d862bb71617e3dbb0fcb8b2fff1696a2bc04fecfbf60b73696: Status 404 returned error can't find the container with id d45de9515f8975d862bb71617e3dbb0fcb8b2fff1696a2bc04fecfbf60b73696
Jul 06 22:32:51.423478 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:32:51.423440    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-9q89m" event={"ID":"53a67b97-ec4a-4505-8fd7-fc6bec48335a","Type":"ContainerStarted","Data":"d45de9515f8975d862bb71617e3dbb0fcb8b2fff1696a2bc04fecfbf60b73696"}
Jul 06 22:32:54.439985 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:32:54.439951    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-9q89m" event={"ID":"53a67b97-ec4a-4505-8fd7-fc6bec48335a","Type":"ContainerStarted","Data":"a4a19bf4b7e7387c9a7dac68ba5819e4c85fd48fefc48faa9ea88c61a04e610f"}
Jul 06 22:32:54.440395 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:32:54.440083    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-9q89m"
Jul 06 22:32:54.456973 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:32:54.456928    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-9q89m" podStartSLOduration=1.702330387 podStartE2EDuration="4.45691425s" totalImagesPullingTime="2.754583863s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:32:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:32:54.454891625 +0000 UTC m=+539.879938179" watchObservedRunningTime="2026-07-06 22:32:54.45691425 +0000 UTC m=+539.881960790"
Jul 06 22:33:05.445407 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:33:05.445377    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-9q89m"
Jul 06 22:33:55.015554 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:33:55.015524    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-t6jpk_03309f33-ff47-447d-956f-c89bdd4f8054/ovn-acl-logging/0.log"
Jul 06 22:33:55.020806 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:33:55.020785    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-t6jpk_03309f33-ff47-447d-956f-c89bdd4f8054/ovn-acl-logging/0.log"
Jul 06 22:34:03.500401 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:34:03.500365    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9q89m"]
Jul 06 22:34:37.069402 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:34:37.069368    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9q89m"]
Jul 06 22:34:59.183718 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:34:59.183681    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9q89m"]
Jul 06 22:35:18.180927 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:35:18.180895    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9q89m"]
Jul 06 22:35:25.080569 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:35:25.080543    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9q89m"]
Jul 06 22:35:36.079549 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:35:36.079518    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-52s95"]
Jul 06 22:35:36.083077 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:35:36.083056    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-52s95"
Jul 06 22:35:36.086594 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:35:36.086574    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 06 22:35:36.086709 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:35:36.086633    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-p5kpm\""
Jul 06 22:35:36.086709 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:35:36.086662    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 06 22:35:36.086709 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:35:36.086679    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Jul 06 22:35:36.093381 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:35:36.093360    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-52s95"]
Jul 06 22:35:36.161015 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:35:36.160978    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/97bcd776-42fd-449d-8194-48acdbbdfb28-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-52s95\" (UID: \"97bcd776-42fd-449d-8194-48acdbbdfb28\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-52s95"
Jul 06 22:35:36.161203 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:35:36.161065    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/97bcd776-42fd-449d-8194-48acdbbdfb28-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-52s95\" (UID: \"97bcd776-42fd-449d-8194-48acdbbdfb28\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-52s95"
Jul 06 22:35:36.161203 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:35:36.161118    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/97bcd776-42fd-449d-8194-48acdbbdfb28-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-52s95\" (UID: \"97bcd776-42fd-449d-8194-48acdbbdfb28\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-52s95"
Jul 06 22:35:36.161374 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:35:36.161309    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/97bcd776-42fd-449d-8194-48acdbbdfb28-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-52s95\" (UID: \"97bcd776-42fd-449d-8194-48acdbbdfb28\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-52s95"
Jul 06 22:35:36.161374 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:35:36.161343    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/97bcd776-42fd-449d-8194-48acdbbdfb28-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-52s95\" (UID: \"97bcd776-42fd-449d-8194-48acdbbdfb28\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-52s95"
Jul 06 22:35:36.161479 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:35:36.161373    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6fz9g\" (UniqueName: \"kubernetes.io/projected/97bcd776-42fd-449d-8194-48acdbbdfb28-kube-api-access-6fz9g\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-52s95\" (UID: \"97bcd776-42fd-449d-8194-48acdbbdfb28\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-52s95"
Jul 06 22:35:36.262303 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:35:36.262268    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/97bcd776-42fd-449d-8194-48acdbbdfb28-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-52s95\" (UID: \"97bcd776-42fd-449d-8194-48acdbbdfb28\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-52s95"
Jul 06 22:35:36.262479 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:35:36.262311    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/97bcd776-42fd-449d-8194-48acdbbdfb28-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-52s95\" (UID: \"97bcd776-42fd-449d-8194-48acdbbdfb28\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-52s95"
Jul 06 22:35:36.262479 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:35:36.262354    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/97bcd776-42fd-449d-8194-48acdbbdfb28-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-52s95\" (UID: \"97bcd776-42fd-449d-8194-48acdbbdfb28\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-52s95"
Jul 06 22:35:36.262479 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:35:36.262369    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/97bcd776-42fd-449d-8194-48acdbbdfb28-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-52s95\" (UID: \"97bcd776-42fd-449d-8194-48acdbbdfb28\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-52s95"
Jul 06 22:35:36.262479 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:35:36.262386    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6fz9g\" (UniqueName: \"kubernetes.io/projected/97bcd776-42fd-449d-8194-48acdbbdfb28-kube-api-access-6fz9g\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-52s95\" (UID: \"97bcd776-42fd-449d-8194-48acdbbdfb28\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-52s95"
Jul 06 22:35:36.262479 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:35:36.262414    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/97bcd776-42fd-449d-8194-48acdbbdfb28-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-52s95\" (UID: \"97bcd776-42fd-449d-8194-48acdbbdfb28\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-52s95"
Jul 06 22:35:36.262819 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:35:36.262796    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/97bcd776-42fd-449d-8194-48acdbbdfb28-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-52s95\" (UID: \"97bcd776-42fd-449d-8194-48acdbbdfb28\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-52s95"
Jul 06 22:35:36.262887 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:35:36.262831    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/97bcd776-42fd-449d-8194-48acdbbdfb28-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-52s95\" (UID: \"97bcd776-42fd-449d-8194-48acdbbdfb28\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-52s95"
Jul 06 22:35:36.262887 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:35:36.262848    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/97bcd776-42fd-449d-8194-48acdbbdfb28-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-52s95\" (UID: \"97bcd776-42fd-449d-8194-48acdbbdfb28\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-52s95"
Jul 06 22:35:36.264715 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:35:36.264696    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/97bcd776-42fd-449d-8194-48acdbbdfb28-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-52s95\" (UID: \"97bcd776-42fd-449d-8194-48acdbbdfb28\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-52s95"
Jul 06 22:35:36.264809 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:35:36.264793    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/97bcd776-42fd-449d-8194-48acdbbdfb28-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-52s95\" (UID: \"97bcd776-42fd-449d-8194-48acdbbdfb28\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-52s95"
Jul 06 22:35:36.270482 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:35:36.270457    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6fz9g\" (UniqueName: \"kubernetes.io/projected/97bcd776-42fd-449d-8194-48acdbbdfb28-kube-api-access-6fz9g\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-52s95\" (UID: \"97bcd776-42fd-449d-8194-48acdbbdfb28\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-52s95"
Jul 06 22:35:36.393375 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:35:36.393239    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-52s95"
Jul 06 22:35:36.549337 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:35:36.549311    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-52s95"]
Jul 06 22:35:36.551935 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:35:36.551899    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97bcd776_42fd_449d_8194_48acdbbdfb28.slice/crio-0cb5ef1ee30bb11b627a482b348412627b6b778158673a45a18dde8861e4721c WatchSource:0}: Error finding container 0cb5ef1ee30bb11b627a482b348412627b6b778158673a45a18dde8861e4721c: Status 404 returned error can't find the container with id 0cb5ef1ee30bb11b627a482b348412627b6b778158673a45a18dde8861e4721c
Jul 06 22:35:36.553551 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:35:36.553532    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 06 22:35:37.161399 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:35:37.161365    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-52s95" event={"ID":"97bcd776-42fd-449d-8194-48acdbbdfb28","Type":"ContainerStarted","Data":"0cb5ef1ee30bb11b627a482b348412627b6b778158673a45a18dde8861e4721c"}
Jul 06 22:35:38.167867 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:35:38.167828    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-52s95" event={"ID":"97bcd776-42fd-449d-8194-48acdbbdfb28","Type":"ContainerStarted","Data":"296eac9dd8eca14422ade0c035c2a4ea526c7ee2008bfdfc95ea48c19b434cef"}
Jul 06 22:35:38.185244 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:35:38.185198    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-52s95" podStartSLOduration=1.070012185 podStartE2EDuration="2.185185212s" totalImagesPullingTime="1.115173027s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:35:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:35:38.183184972 +0000 UTC m=+703.608231506" watchObservedRunningTime="2026-07-06 22:35:38.185185212 +0000 UTC m=+703.610231744"
Jul 06 22:35:38.477846 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:35:38.477811    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9q89m"]
Jul 06 22:35:46.393616 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:35:46.393534    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-52s95"
Jul 06 22:35:46.393616 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:35:46.393594    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-52s95"
Jul 06 22:35:46.406396 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:35:46.406369    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-52s95"
Jul 06 22:35:47.221256 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:35:47.221230    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-52s95"
Jul 06 22:35:54.479483 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:35:54.479451    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9q89m"]
Jul 06 22:37:24.776704 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:37:24.776621    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9q89m"]
Jul 06 22:37:34.980889 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:37:34.980852    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9q89m"]
Jul 06 22:37:45.566457 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:37:45.566421    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9q89m"]
Jul 06 22:37:55.576216 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:37:55.576165    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9q89m"]
Jul 06 22:38:05.570012 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:38:05.569970    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9q89m"]
Jul 06 22:38:09.719212 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:38:09.719163    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6dccc7f4ff-4vrzw"]
Jul 06 22:38:09.727920 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:38:09.727889    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6dccc7f4ff-4vrzw"
Jul 06 22:38:09.731441 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:38:09.731416    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 06 22:38:09.731628 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:38:09.731432    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 06 22:38:09.731839 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:38:09.731476    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-722xt\""
Jul 06 22:38:09.733769 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:38:09.733750    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6dccc7f4ff-4vrzw"]
Jul 06 22:38:09.805512 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:38:09.805483    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9d2b9b0b-ce4e-431d-b762-87dc80ad88a0-webhook-cert\") pod \"maas-controller-6dccc7f4ff-4vrzw\" (UID: \"9d2b9b0b-ce4e-431d-b762-87dc80ad88a0\") " pod="opendatahub/maas-controller-6dccc7f4ff-4vrzw"
Jul 06 22:38:09.805512 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:38:09.805512    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vs5v9\" (UniqueName: \"kubernetes.io/projected/9d2b9b0b-ce4e-431d-b762-87dc80ad88a0-kube-api-access-vs5v9\") pod \"maas-controller-6dccc7f4ff-4vrzw\" (UID: \"9d2b9b0b-ce4e-431d-b762-87dc80ad88a0\") " pod="opendatahub/maas-controller-6dccc7f4ff-4vrzw"
Jul 06 22:38:09.906659 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:38:09.906624    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9d2b9b0b-ce4e-431d-b762-87dc80ad88a0-webhook-cert\") pod \"maas-controller-6dccc7f4ff-4vrzw\" (UID: \"9d2b9b0b-ce4e-431d-b762-87dc80ad88a0\") " pod="opendatahub/maas-controller-6dccc7f4ff-4vrzw"
Jul 06 22:38:09.906836 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:38:09.906663    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vs5v9\" (UniqueName: \"kubernetes.io/projected/9d2b9b0b-ce4e-431d-b762-87dc80ad88a0-kube-api-access-vs5v9\") pod \"maas-controller-6dccc7f4ff-4vrzw\" (UID: \"9d2b9b0b-ce4e-431d-b762-87dc80ad88a0\") " pod="opendatahub/maas-controller-6dccc7f4ff-4vrzw"
Jul 06 22:38:09.909194 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:38:09.909156    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9d2b9b0b-ce4e-431d-b762-87dc80ad88a0-webhook-cert\") pod \"maas-controller-6dccc7f4ff-4vrzw\" (UID: \"9d2b9b0b-ce4e-431d-b762-87dc80ad88a0\") " pod="opendatahub/maas-controller-6dccc7f4ff-4vrzw"
Jul 06 22:38:09.916162 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:38:09.916129    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vs5v9\" (UniqueName: \"kubernetes.io/projected/9d2b9b0b-ce4e-431d-b762-87dc80ad88a0-kube-api-access-vs5v9\") pod \"maas-controller-6dccc7f4ff-4vrzw\" (UID: \"9d2b9b0b-ce4e-431d-b762-87dc80ad88a0\") " pod="opendatahub/maas-controller-6dccc7f4ff-4vrzw"
Jul 06 22:38:10.040448 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:38:10.040367    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6dccc7f4ff-4vrzw"
Jul 06 22:38:10.173136 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:38:10.173107    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6dccc7f4ff-4vrzw"]
Jul 06 22:38:10.176359 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:38:10.176328    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d2b9b0b_ce4e_431d_b762_87dc80ad88a0.slice/crio-97c55801a4c7aabd302fd11933b347a8a8902cd80238ce79a64de48ad01c2955 WatchSource:0}: Error finding container 97c55801a4c7aabd302fd11933b347a8a8902cd80238ce79a64de48ad01c2955: Status 404 returned error can't find the container with id 97c55801a4c7aabd302fd11933b347a8a8902cd80238ce79a64de48ad01c2955
Jul 06 22:38:10.844633 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:38:10.844589    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6dccc7f4ff-4vrzw" event={"ID":"9d2b9b0b-ce4e-431d-b762-87dc80ad88a0","Type":"ContainerStarted","Data":"97c55801a4c7aabd302fd11933b347a8a8902cd80238ce79a64de48ad01c2955"}
Jul 06 22:38:12.857110 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:38:12.857075    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6dccc7f4ff-4vrzw" event={"ID":"9d2b9b0b-ce4e-431d-b762-87dc80ad88a0","Type":"ContainerStarted","Data":"374b105b89cd8dbbcb77706c92887e7f12fd4d5b50b2584b86748080f1fe94dd"}
Jul 06 22:38:12.857505 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:38:12.857221    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6dccc7f4ff-4vrzw"
Jul 06 22:38:12.874562 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:38:12.874474    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6dccc7f4ff-4vrzw" podStartSLOduration=1.4264712 podStartE2EDuration="3.87445984s" totalImagesPullingTime="2.44798864s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:38:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:38:12.873729369 +0000 UTC m=+858.298775911" watchObservedRunningTime="2026-07-06 22:38:12.87445984 +0000 UTC m=+858.299506381"
Jul 06 22:38:23.868982 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:38:23.868951    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6dccc7f4ff-4vrzw"
Jul 06 22:38:35.570648 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:38:35.570614    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9q89m"]
Jul 06 22:38:49.378320 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:38:49.378242    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9q89m"]
Jul 06 22:38:55.045343 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:38:55.045313    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-t6jpk_03309f33-ff47-447d-956f-c89bdd4f8054/ovn-acl-logging/0.log"
Jul 06 22:38:55.046637 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:38:55.046618    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-t6jpk_03309f33-ff47-447d-956f-c89bdd4f8054/ovn-acl-logging/0.log"
Jul 06 22:39:24.171418 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:39:24.171383    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9q89m"]
Jul 06 22:39:33.770655 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:39:33.770623    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9q89m"]
Jul 06 22:39:42.675210 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:39:42.675162    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9q89m"]
Jul 06 22:39:51.665121 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:39:51.665087    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9q89m"]
Jul 06 22:40:00.675040 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:40:00.675007    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9q89m"]
Jul 06 22:40:11.777503 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:40:11.777468    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9q89m"]
Jul 06 22:40:22.770701 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:40:22.770621    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9q89m"]
Jul 06 22:40:32.574058 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:40:32.574023    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9q89m"]
Jul 06 22:40:55.382726 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:40:55.382684    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9q89m"]
Jul 06 22:41:04.179249 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:41:04.179207    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9q89m"]
Jul 06 22:41:26.866901 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:41:26.866865    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9q89m"]
Jul 06 22:41:36.479316 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:41:36.479280    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9q89m"]
Jul 06 22:41:54.883277 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:41:54.883199    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9q89m"]
Jul 06 22:42:04.075652 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:42:04.075620    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9q89m"]
Jul 06 22:42:13.673963 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:42:13.673927    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9q89m"]
Jul 06 22:42:22.875093 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:42:22.875059    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9q89m"]
Jul 06 22:42:30.683950 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:42:30.683921    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9q89m"]
Jul 06 22:42:58.139106 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:42:58.139063    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-h559w"]
Jul 06 22:42:58.139770 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:42:58.139676    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-h559w" podUID="782bd269-b2e7-4411-b3d9-39bf3466cfb2" containerName="manager" containerID="cri-o://af0d5de136b279b2c867dc5dd3785005acdde4e970f29497a1ae8976ec2f286b" gracePeriod=10
Jul 06 22:43:01.336390 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:43:01.336352    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-h559w" podUID="782bd269-b2e7-4411-b3d9-39bf3466cfb2" containerName="manager" probeResult="failure" output="Get \"http://10.133.0.29:8081/readyz\": dial tcp 10.133.0.29:8081: connect: connection refused"
Jul 06 22:43:01.487239 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:43:01.487216    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-h559w"
Jul 06 22:43:01.551911 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:43:01.551880    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/782bd269-b2e7-4411-b3d9-39bf3466cfb2-extensions-socket-volume\") pod \"782bd269-b2e7-4411-b3d9-39bf3466cfb2\" (UID: \"782bd269-b2e7-4411-b3d9-39bf3466cfb2\") "
Jul 06 22:43:01.552077 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:43:01.551925    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-88xqb\" (UniqueName: \"kubernetes.io/projected/782bd269-b2e7-4411-b3d9-39bf3466cfb2-kube-api-access-88xqb\") pod \"782bd269-b2e7-4411-b3d9-39bf3466cfb2\" (UID: \"782bd269-b2e7-4411-b3d9-39bf3466cfb2\") "
Jul 06 22:43:01.552283 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:43:01.552259    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/782bd269-b2e7-4411-b3d9-39bf3466cfb2-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "782bd269-b2e7-4411-b3d9-39bf3466cfb2" (UID: "782bd269-b2e7-4411-b3d9-39bf3466cfb2"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 22:43:01.554082 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:43:01.554058    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/782bd269-b2e7-4411-b3d9-39bf3466cfb2-kube-api-access-88xqb" (OuterVolumeSpecName: "kube-api-access-88xqb") pod "782bd269-b2e7-4411-b3d9-39bf3466cfb2" (UID: "782bd269-b2e7-4411-b3d9-39bf3466cfb2"). InnerVolumeSpecName "kube-api-access-88xqb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 22:43:01.653081 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:43:01.652995    2563 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/782bd269-b2e7-4411-b3d9-39bf3466cfb2-extensions-socket-volume\") on node \"ip-10-0-133-105.ec2.internal\" DevicePath \"\""
Jul 06 22:43:01.653081 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:43:01.653026    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-88xqb\" (UniqueName: \"kubernetes.io/projected/782bd269-b2e7-4411-b3d9-39bf3466cfb2-kube-api-access-88xqb\") on node \"ip-10-0-133-105.ec2.internal\" DevicePath \"\""
Jul 06 22:43:02.174306 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:43:02.174270    2563 generic.go:356] "Generic (PLEG): container finished" podID="782bd269-b2e7-4411-b3d9-39bf3466cfb2" containerID="af0d5de136b279b2c867dc5dd3785005acdde4e970f29497a1ae8976ec2f286b" exitCode=0
Jul 06 22:43:02.174483 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:43:02.174314    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-h559w"
Jul 06 22:43:02.174483 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:43:02.174351    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-h559w" event={"ID":"782bd269-b2e7-4411-b3d9-39bf3466cfb2","Type":"ContainerDied","Data":"af0d5de136b279b2c867dc5dd3785005acdde4e970f29497a1ae8976ec2f286b"}
Jul 06 22:43:02.174483 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:43:02.174406    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-h559w" event={"ID":"782bd269-b2e7-4411-b3d9-39bf3466cfb2","Type":"ContainerDied","Data":"73cdb3057f2fd7e6b3d23c2bb48cf2f6c9342090ead6b2cd5113af24e2d21bbe"}
Jul 06 22:43:02.174483 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:43:02.174428    2563 scope.go:117] "RemoveContainer" containerID="af0d5de136b279b2c867dc5dd3785005acdde4e970f29497a1ae8976ec2f286b"
Jul 06 22:43:02.187322 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:43:02.187305    2563 scope.go:117] "RemoveContainer" containerID="af0d5de136b279b2c867dc5dd3785005acdde4e970f29497a1ae8976ec2f286b"
Jul 06 22:43:02.187602 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:43:02.187585    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"af0d5de136b279b2c867dc5dd3785005acdde4e970f29497a1ae8976ec2f286b\": container with ID starting with af0d5de136b279b2c867dc5dd3785005acdde4e970f29497a1ae8976ec2f286b not found: ID does not exist" containerID="af0d5de136b279b2c867dc5dd3785005acdde4e970f29497a1ae8976ec2f286b"
Jul 06 22:43:02.187656 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:43:02.187611    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"af0d5de136b279b2c867dc5dd3785005acdde4e970f29497a1ae8976ec2f286b"} err="failed to get container status \"af0d5de136b279b2c867dc5dd3785005acdde4e970f29497a1ae8976ec2f286b\": rpc error: code = NotFound desc = could not find container \"af0d5de136b279b2c867dc5dd3785005acdde4e970f29497a1ae8976ec2f286b\": container with ID starting with af0d5de136b279b2c867dc5dd3785005acdde4e970f29497a1ae8976ec2f286b not found: ID does not exist"
Jul 06 22:43:02.197395 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:43:02.197367    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-h559w"]
Jul 06 22:43:02.202918 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:43:02.202895    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-h559w"]
Jul 06 22:43:03.080435 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:43:03.080401    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="782bd269-b2e7-4411-b3d9-39bf3466cfb2" path="/var/lib/kubelet/pods/782bd269-b2e7-4411-b3d9-39bf3466cfb2/volumes"
Jul 06 22:43:55.071960 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:43:55.071934    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-t6jpk_03309f33-ff47-447d-956f-c89bdd4f8054/ovn-acl-logging/0.log"
Jul 06 22:43:55.072551 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:43:55.072014    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-t6jpk_03309f33-ff47-447d-956f-c89bdd4f8054/ovn-acl-logging/0.log"
Jul 06 22:43:59.267613 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:43:59.267584    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fcqxm"]
Jul 06 22:43:59.267970 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:43:59.267929    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="782bd269-b2e7-4411-b3d9-39bf3466cfb2" containerName="manager"
Jul 06 22:43:59.267970 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:43:59.267939    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="782bd269-b2e7-4411-b3d9-39bf3466cfb2" containerName="manager"
Jul 06 22:43:59.268042 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:43:59.267997    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="782bd269-b2e7-4411-b3d9-39bf3466cfb2" containerName="manager"
Jul 06 22:43:59.271123 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:43:59.271104    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fcqxm"
Jul 06 22:43:59.273912 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:43:59.273891    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-bzlzn\""
Jul 06 22:43:59.283428 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:43:59.283405    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fcqxm"]
Jul 06 22:43:59.347932 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:43:59.342881    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/dff0d5e7-693e-498e-8ffe-fadbfff8635f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-fcqxm\" (UID: \"dff0d5e7-693e-498e-8ffe-fadbfff8635f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fcqxm"
Jul 06 22:43:59.347932 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:43:59.343016    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l6l9c\" (UniqueName: \"kubernetes.io/projected/dff0d5e7-693e-498e-8ffe-fadbfff8635f-kube-api-access-l6l9c\") pod \"kuadrant-operator-controller-manager-86878c769f-fcqxm\" (UID: \"dff0d5e7-693e-498e-8ffe-fadbfff8635f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fcqxm"
Jul 06 22:43:59.444091 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:43:59.444059    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/dff0d5e7-693e-498e-8ffe-fadbfff8635f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-fcqxm\" (UID: \"dff0d5e7-693e-498e-8ffe-fadbfff8635f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fcqxm"
Jul 06 22:43:59.444091 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:43:59.444094    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l6l9c\" (UniqueName: \"kubernetes.io/projected/dff0d5e7-693e-498e-8ffe-fadbfff8635f-kube-api-access-l6l9c\") pod \"kuadrant-operator-controller-manager-86878c769f-fcqxm\" (UID: \"dff0d5e7-693e-498e-8ffe-fadbfff8635f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fcqxm"
Jul 06 22:43:59.444501 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:43:59.444477    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/dff0d5e7-693e-498e-8ffe-fadbfff8635f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-fcqxm\" (UID: \"dff0d5e7-693e-498e-8ffe-fadbfff8635f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fcqxm"
Jul 06 22:43:59.453222 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:43:59.453196    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l6l9c\" (UniqueName: \"kubernetes.io/projected/dff0d5e7-693e-498e-8ffe-fadbfff8635f-kube-api-access-l6l9c\") pod \"kuadrant-operator-controller-manager-86878c769f-fcqxm\" (UID: \"dff0d5e7-693e-498e-8ffe-fadbfff8635f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fcqxm"
Jul 06 22:43:59.581750 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:43:59.581664    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fcqxm"
Jul 06 22:43:59.727665 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:43:59.727640    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fcqxm"]
Jul 06 22:43:59.729941 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:43:59.729910    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddff0d5e7_693e_498e_8ffe_fadbfff8635f.slice/crio-a30e3b512606697c90085e7509461ac6c4ccfbe0f640f3638817c20e4462b320 WatchSource:0}: Error finding container a30e3b512606697c90085e7509461ac6c4ccfbe0f640f3638817c20e4462b320: Status 404 returned error can't find the container with id a30e3b512606697c90085e7509461ac6c4ccfbe0f640f3638817c20e4462b320
Jul 06 22:44:00.440887 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:44:00.440853    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fcqxm" event={"ID":"dff0d5e7-693e-498e-8ffe-fadbfff8635f","Type":"ContainerStarted","Data":"fde7cea8219584cbf60b1e65d6d350742147c0dfe42d7aa0cf1947c2feda1447"}
Jul 06 22:44:00.441312 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:44:00.440926    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fcqxm" event={"ID":"dff0d5e7-693e-498e-8ffe-fadbfff8635f","Type":"ContainerStarted","Data":"a30e3b512606697c90085e7509461ac6c4ccfbe0f640f3638817c20e4462b320"}
Jul 06 22:44:00.441312 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:44:00.440954    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fcqxm"
Jul 06 22:44:00.470144 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:44:00.470094    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fcqxm" podStartSLOduration=1.47007865 podStartE2EDuration="1.47007865s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:43:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:44:00.469302355 +0000 UTC m=+1205.894348904" watchObservedRunningTime="2026-07-06 22:44:00.47007865 +0000 UTC m=+1205.895125191"
Jul 06 22:44:11.447696 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:44:11.447670    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fcqxm"
Jul 06 22:44:20.973445 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:44:20.973407    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9q89m"]
Jul 06 22:44:29.871225 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:44:29.871183    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9q89m"]
Jul 06 22:44:42.976759 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:44:42.976721    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9q89m"]
Jul 06 22:44:53.773418 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:44:53.773345    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9q89m"]
Jul 06 22:45:00.138212 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:45:00.138164    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-29722965-ht74g"]
Jul 06 22:45:00.141540 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:45:00.141524    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29722965-ht74g"
Jul 06 22:45:00.144156 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:45:00.144127    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-f7f72\""
Jul 06 22:45:00.150022 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:45:00.149999    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29722965-ht74g"]
Jul 06 22:45:00.281315 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:45:00.281273    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qhbk6\" (UniqueName: \"kubernetes.io/projected/f0e88425-ee44-47d8-bac2-df42a5582f64-kube-api-access-qhbk6\") pod \"maas-api-key-cleanup-29722965-ht74g\" (UID: \"f0e88425-ee44-47d8-bac2-df42a5582f64\") " pod="opendatahub/maas-api-key-cleanup-29722965-ht74g"
Jul 06 22:45:00.382804 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:45:00.382766    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qhbk6\" (UniqueName: \"kubernetes.io/projected/f0e88425-ee44-47d8-bac2-df42a5582f64-kube-api-access-qhbk6\") pod \"maas-api-key-cleanup-29722965-ht74g\" (UID: \"f0e88425-ee44-47d8-bac2-df42a5582f64\") " pod="opendatahub/maas-api-key-cleanup-29722965-ht74g"
Jul 06 22:45:00.396445 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:45:00.396376    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qhbk6\" (UniqueName: \"kubernetes.io/projected/f0e88425-ee44-47d8-bac2-df42a5582f64-kube-api-access-qhbk6\") pod \"maas-api-key-cleanup-29722965-ht74g\" (UID: \"f0e88425-ee44-47d8-bac2-df42a5582f64\") " pod="opendatahub/maas-api-key-cleanup-29722965-ht74g"
Jul 06 22:45:00.453036 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:45:00.453002    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29722965-ht74g"
Jul 06 22:45:00.587745 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:45:00.587597    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29722965-ht74g"]
Jul 06 22:45:00.590681 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:45:00.590647    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0e88425_ee44_47d8_bac2_df42a5582f64.slice/crio-1845fe62bb4a2e3f9bfae873d32142cb453c2703aa9a159ce7636e7c19087615 WatchSource:0}: Error finding container 1845fe62bb4a2e3f9bfae873d32142cb453c2703aa9a159ce7636e7c19087615: Status 404 returned error can't find the container with id 1845fe62bb4a2e3f9bfae873d32142cb453c2703aa9a159ce7636e7c19087615
Jul 06 22:45:00.592634 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:45:00.592618    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 06 22:45:00.722260 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:45:00.722227    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29722965-ht74g" event={"ID":"f0e88425-ee44-47d8-bac2-df42a5582f64","Type":"ContainerStarted","Data":"1845fe62bb4a2e3f9bfae873d32142cb453c2703aa9a159ce7636e7c19087615"}
Jul 06 22:45:03.739409 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:45:03.739379    2563 generic.go:356] "Generic (PLEG): container finished" podID="f0e88425-ee44-47d8-bac2-df42a5582f64" containerID="71bd532ec230baf09ab2fc0d4fb520d878294aee4301fe6f50a4693e9bca067b" exitCode=0
Jul 06 22:45:03.739720 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:45:03.739460    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29722965-ht74g" event={"ID":"f0e88425-ee44-47d8-bac2-df42a5582f64","Type":"ContainerDied","Data":"71bd532ec230baf09ab2fc0d4fb520d878294aee4301fe6f50a4693e9bca067b"}
Jul 06 22:45:04.878719 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:45:04.878688    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29722965-ht74g"
Jul 06 22:45:05.025585 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:45:05.025500    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qhbk6\" (UniqueName: \"kubernetes.io/projected/f0e88425-ee44-47d8-bac2-df42a5582f64-kube-api-access-qhbk6\") pod \"f0e88425-ee44-47d8-bac2-df42a5582f64\" (UID: \"f0e88425-ee44-47d8-bac2-df42a5582f64\") "
Jul 06 22:45:05.027718 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:45:05.027686    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f0e88425-ee44-47d8-bac2-df42a5582f64-kube-api-access-qhbk6" (OuterVolumeSpecName: "kube-api-access-qhbk6") pod "f0e88425-ee44-47d8-bac2-df42a5582f64" (UID: "f0e88425-ee44-47d8-bac2-df42a5582f64"). InnerVolumeSpecName "kube-api-access-qhbk6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 22:45:05.126056 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:45:05.126024    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qhbk6\" (UniqueName: \"kubernetes.io/projected/f0e88425-ee44-47d8-bac2-df42a5582f64-kube-api-access-qhbk6\") on node \"ip-10-0-133-105.ec2.internal\" DevicePath \"\""
Jul 06 22:45:05.180160 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:45:05.180127    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9q89m"]
Jul 06 22:45:05.751156 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:45:05.751123    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29722965-ht74g"
Jul 06 22:45:05.751353 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:45:05.751147    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29722965-ht74g" event={"ID":"f0e88425-ee44-47d8-bac2-df42a5582f64","Type":"ContainerDied","Data":"1845fe62bb4a2e3f9bfae873d32142cb453c2703aa9a159ce7636e7c19087615"}
Jul 06 22:45:05.751353 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:45:05.751315    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1845fe62bb4a2e3f9bfae873d32142cb453c2703aa9a159ce7636e7c19087615"
Jul 06 22:45:14.492273 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:45:14.492235    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9q89m"]
Jul 06 22:45:24.567044 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:45:24.567004    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9q89m"]
Jul 06 22:45:35.174670 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:45:35.174632    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9q89m"]
Jul 06 22:45:46.270708 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:45:46.270674    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9q89m"]
Jul 06 22:45:54.875633 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:45:54.875592    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9q89m"]
Jul 06 22:46:05.068549 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:46:05.068514    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9q89m"]
Jul 06 22:46:27.670278 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:46:27.670240    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9q89m"]
Jul 06 22:46:36.770712 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:46:36.770679    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9q89m"]
Jul 06 22:46:46.170998 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:46:46.170955    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9q89m"]
Jul 06 22:46:57.270132 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:46:57.270094    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9q89m"]
Jul 06 22:47:04.976856 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:47:04.976825    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9q89m"]
Jul 06 22:47:15.674844 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:47:15.674807    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9q89m"]
Jul 06 22:47:24.979363 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:47:24.979328    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9q89m"]
Jul 06 22:47:34.677733 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:47:34.677701    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9q89m"]
Jul 06 22:47:42.475925 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:47:42.475892    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9q89m"]
Jul 06 22:47:52.984680 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:47:52.984598    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9q89m"]
Jul 06 22:48:02.371194 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:48:02.371136    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9q89m"]
Jul 06 22:48:14.980909 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:48:14.980874    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9q89m"]
Jul 06 22:48:24.854512 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:48:24.854472    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9q89m"]
Jul 06 22:48:34.369123 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:48:34.369085    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9q89m"]
Jul 06 22:48:43.580195 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:48:43.580148    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9q89m"]
Jul 06 22:48:55.099954 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:48:55.099923    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-t6jpk_03309f33-ff47-447d-956f-c89bdd4f8054/ovn-acl-logging/0.log"
Jul 06 22:48:55.100562 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:48:55.100219    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-t6jpk_03309f33-ff47-447d-956f-c89bdd4f8054/ovn-acl-logging/0.log"
Jul 06 22:48:58.678958 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:48:58.678923    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9q89m"]
Jul 06 22:49:06.385407 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:49:06.385372    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9q89m"]
Jul 06 22:49:18.672216 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:49:18.672125    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9q89m"]
Jul 06 22:49:30.882220 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:49:30.882163    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9q89m"]
Jul 06 22:49:43.977339 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:49:43.977306    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9q89m"]
Jul 06 22:49:56.674390 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:49:56.674359    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9q89m"]
Jul 06 22:50:05.871554 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:50:05.871515    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9q89m"]
Jul 06 22:50:17.379160 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:50:17.379125    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9q89m"]
Jul 06 22:50:18.029326 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:50:18.029287    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-ait-ba18ff93-5ff4544976-zrlbp"]
Jul 06 22:50:18.029894 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:50:18.029868    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f0e88425-ee44-47d8-bac2-df42a5582f64" containerName="cleanup"
Jul 06 22:50:18.029894 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:50:18.029895    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="f0e88425-ee44-47d8-bac2-df42a5582f64" containerName="cleanup"
Jul 06 22:50:18.030108 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:50:18.029985    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="f0e88425-ee44-47d8-bac2-df42a5582f64" containerName="cleanup"
Jul 06 22:50:18.032924 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:50:18.032903    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-ba18ff93-5ff4544976-zrlbp"
Jul 06 22:50:18.035423 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:50:18.035400    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-ait-ba18ff93-serving-cert\""
Jul 06 22:50:18.035547 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:50:18.035469    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-f7f72\""
Jul 06 22:50:18.044135 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:50:18.044110    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-ba18ff93-5ff4544976-zrlbp"]
Jul 06 22:50:18.190609 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:50:18.190562    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f5b2ae39-43d8-4773-b1e9-e9734ac6ca0c-openshift-service-ca\") pod \"maas-api-e2e-ait-ba18ff93-5ff4544976-zrlbp\" (UID: \"f5b2ae39-43d8-4773-b1e9-e9734ac6ca0c\") " pod="opendatahub/maas-api-e2e-ait-ba18ff93-5ff4544976-zrlbp"
Jul 06 22:50:18.190796 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:50:18.190646    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p4zz2\" (UniqueName: \"kubernetes.io/projected/f5b2ae39-43d8-4773-b1e9-e9734ac6ca0c-kube-api-access-p4zz2\") pod \"maas-api-e2e-ait-ba18ff93-5ff4544976-zrlbp\" (UID: \"f5b2ae39-43d8-4773-b1e9-e9734ac6ca0c\") " pod="opendatahub/maas-api-e2e-ait-ba18ff93-5ff4544976-zrlbp"
Jul 06 22:50:18.190796 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:50:18.190764    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f5b2ae39-43d8-4773-b1e9-e9734ac6ca0c-maas-api-tls\") pod \"maas-api-e2e-ait-ba18ff93-5ff4544976-zrlbp\" (UID: \"f5b2ae39-43d8-4773-b1e9-e9734ac6ca0c\") " pod="opendatahub/maas-api-e2e-ait-ba18ff93-5ff4544976-zrlbp"
Jul 06 22:50:18.292313 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:50:18.292224    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f5b2ae39-43d8-4773-b1e9-e9734ac6ca0c-openshift-service-ca\") pod \"maas-api-e2e-ait-ba18ff93-5ff4544976-zrlbp\" (UID: \"f5b2ae39-43d8-4773-b1e9-e9734ac6ca0c\") " pod="opendatahub/maas-api-e2e-ait-ba18ff93-5ff4544976-zrlbp"
Jul 06 22:50:18.292313 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:50:18.292266    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p4zz2\" (UniqueName: \"kubernetes.io/projected/f5b2ae39-43d8-4773-b1e9-e9734ac6ca0c-kube-api-access-p4zz2\") pod \"maas-api-e2e-ait-ba18ff93-5ff4544976-zrlbp\" (UID: \"f5b2ae39-43d8-4773-b1e9-e9734ac6ca0c\") " pod="opendatahub/maas-api-e2e-ait-ba18ff93-5ff4544976-zrlbp"
Jul 06 22:50:18.292313 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:50:18.292310    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f5b2ae39-43d8-4773-b1e9-e9734ac6ca0c-maas-api-tls\") pod \"maas-api-e2e-ait-ba18ff93-5ff4544976-zrlbp\" (UID: \"f5b2ae39-43d8-4773-b1e9-e9734ac6ca0c\") " pod="opendatahub/maas-api-e2e-ait-ba18ff93-5ff4544976-zrlbp"
Jul 06 22:50:18.292913 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:50:18.292883    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f5b2ae39-43d8-4773-b1e9-e9734ac6ca0c-openshift-service-ca\") pod \"maas-api-e2e-ait-ba18ff93-5ff4544976-zrlbp\" (UID: \"f5b2ae39-43d8-4773-b1e9-e9734ac6ca0c\") " pod="opendatahub/maas-api-e2e-ait-ba18ff93-5ff4544976-zrlbp"
Jul 06 22:50:18.294752 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:50:18.294733    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f5b2ae39-43d8-4773-b1e9-e9734ac6ca0c-maas-api-tls\") pod \"maas-api-e2e-ait-ba18ff93-5ff4544976-zrlbp\" (UID: \"f5b2ae39-43d8-4773-b1e9-e9734ac6ca0c\") " pod="opendatahub/maas-api-e2e-ait-ba18ff93-5ff4544976-zrlbp"
Jul 06 22:50:18.302666 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:50:18.302642    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p4zz2\" (UniqueName: \"kubernetes.io/projected/f5b2ae39-43d8-4773-b1e9-e9734ac6ca0c-kube-api-access-p4zz2\") pod \"maas-api-e2e-ait-ba18ff93-5ff4544976-zrlbp\" (UID: \"f5b2ae39-43d8-4773-b1e9-e9734ac6ca0c\") " pod="opendatahub/maas-api-e2e-ait-ba18ff93-5ff4544976-zrlbp"
Jul 06 22:50:18.347285 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:50:18.347258    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-ba18ff93-5ff4544976-zrlbp"
Jul 06 22:50:18.489293 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:50:18.489266    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-ba18ff93-5ff4544976-zrlbp"]
Jul 06 22:50:18.491305 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:50:18.491277    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5b2ae39_43d8_4773_b1e9_e9734ac6ca0c.slice/crio-de16537fc893eed4915059f9157c7ab3f1f41687e445ccf8520c97e29449e7e5 WatchSource:0}: Error finding container de16537fc893eed4915059f9157c7ab3f1f41687e445ccf8520c97e29449e7e5: Status 404 returned error can't find the container with id de16537fc893eed4915059f9157c7ab3f1f41687e445ccf8520c97e29449e7e5
Jul 06 22:50:18.492667 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:50:18.492649    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 06 22:50:19.198081 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:50:19.198041    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-ba18ff93-5ff4544976-zrlbp" event={"ID":"f5b2ae39-43d8-4773-b1e9-e9734ac6ca0c","Type":"ContainerStarted","Data":"de16537fc893eed4915059f9157c7ab3f1f41687e445ccf8520c97e29449e7e5"}
Jul 06 22:50:20.204430 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:50:20.204399    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-ba18ff93-5ff4544976-zrlbp" event={"ID":"f5b2ae39-43d8-4773-b1e9-e9734ac6ca0c","Type":"ContainerStarted","Data":"e7461c6dbbafe0e607f817884660b3a4130585aea8cbb1e6d0dc1b8ee048d9ab"}
Jul 06 22:50:20.204810 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:50:20.204534    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-ait-ba18ff93-5ff4544976-zrlbp"
Jul 06 22:50:20.219737 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:50:20.219573    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-ait-ba18ff93-5ff4544976-zrlbp" podStartSLOduration=0.60408013 podStartE2EDuration="2.219553652s" totalImagesPullingTime="1.615473522s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:50:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:50:20.219045751 +0000 UTC m=+1585.644092292" watchObservedRunningTime="2026-07-06 22:50:20.219553652 +0000 UTC m=+1585.644600194"
Jul 06 22:50:23.430212 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:50:23.430182    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-ba18ff93-5ff4544976-zrlbp"]
Jul 06 22:50:23.430665 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:50:23.430378    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-ait-ba18ff93-5ff4544976-zrlbp" podUID="f5b2ae39-43d8-4773-b1e9-e9734ac6ca0c" containerName="maas-api" containerID="cri-o://e7461c6dbbafe0e607f817884660b3a4130585aea8cbb1e6d0dc1b8ee048d9ab" gracePeriod=30
Jul 06 22:50:23.689228 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:50:23.689136    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-ba18ff93-5ff4544976-zrlbp"
Jul 06 22:50:23.738670 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:50:23.738639    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f5b2ae39-43d8-4773-b1e9-e9734ac6ca0c-maas-api-tls\") pod \"f5b2ae39-43d8-4773-b1e9-e9734ac6ca0c\" (UID: \"f5b2ae39-43d8-4773-b1e9-e9734ac6ca0c\") "
Jul 06 22:50:23.738869 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:50:23.738686    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f5b2ae39-43d8-4773-b1e9-e9734ac6ca0c-openshift-service-ca\") pod \"f5b2ae39-43d8-4773-b1e9-e9734ac6ca0c\" (UID: \"f5b2ae39-43d8-4773-b1e9-e9734ac6ca0c\") "
Jul 06 22:50:23.738869 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:50:23.738739    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p4zz2\" (UniqueName: \"kubernetes.io/projected/f5b2ae39-43d8-4773-b1e9-e9734ac6ca0c-kube-api-access-p4zz2\") pod \"f5b2ae39-43d8-4773-b1e9-e9734ac6ca0c\" (UID: \"f5b2ae39-43d8-4773-b1e9-e9734ac6ca0c\") "
Jul 06 22:50:23.739185 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:50:23.739139    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f5b2ae39-43d8-4773-b1e9-e9734ac6ca0c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f5b2ae39-43d8-4773-b1e9-e9734ac6ca0c" (UID: "f5b2ae39-43d8-4773-b1e9-e9734ac6ca0c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 22:50:23.740887 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:50:23.740865    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f5b2ae39-43d8-4773-b1e9-e9734ac6ca0c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f5b2ae39-43d8-4773-b1e9-e9734ac6ca0c" (UID: "f5b2ae39-43d8-4773-b1e9-e9734ac6ca0c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 22:50:23.740984 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:50:23.740964    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f5b2ae39-43d8-4773-b1e9-e9734ac6ca0c-kube-api-access-p4zz2" (OuterVolumeSpecName: "kube-api-access-p4zz2") pod "f5b2ae39-43d8-4773-b1e9-e9734ac6ca0c" (UID: "f5b2ae39-43d8-4773-b1e9-e9734ac6ca0c"). InnerVolumeSpecName "kube-api-access-p4zz2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 22:50:23.839396 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:50:23.839357    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f5b2ae39-43d8-4773-b1e9-e9734ac6ca0c-openshift-service-ca\") on node \"ip-10-0-133-105.ec2.internal\" DevicePath \"\""
Jul 06 22:50:23.839396 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:50:23.839388    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p4zz2\" (UniqueName: \"kubernetes.io/projected/f5b2ae39-43d8-4773-b1e9-e9734ac6ca0c-kube-api-access-p4zz2\") on node \"ip-10-0-133-105.ec2.internal\" DevicePath \"\""
Jul 06 22:50:23.839396 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:50:23.839398    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f5b2ae39-43d8-4773-b1e9-e9734ac6ca0c-maas-api-tls\") on node \"ip-10-0-133-105.ec2.internal\" DevicePath \"\""
Jul 06 22:50:24.223933 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:50:24.223891    2563 generic.go:356] "Generic (PLEG): container finished" podID="f5b2ae39-43d8-4773-b1e9-e9734ac6ca0c" containerID="e7461c6dbbafe0e607f817884660b3a4130585aea8cbb1e6d0dc1b8ee048d9ab" exitCode=0
Jul 06 22:50:24.223933 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:50:24.223935    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-ba18ff93-5ff4544976-zrlbp"
Jul 06 22:50:24.224187 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:50:24.223938    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-ba18ff93-5ff4544976-zrlbp" event={"ID":"f5b2ae39-43d8-4773-b1e9-e9734ac6ca0c","Type":"ContainerDied","Data":"e7461c6dbbafe0e607f817884660b3a4130585aea8cbb1e6d0dc1b8ee048d9ab"}
Jul 06 22:50:24.224187 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:50:24.223994    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-ba18ff93-5ff4544976-zrlbp" event={"ID":"f5b2ae39-43d8-4773-b1e9-e9734ac6ca0c","Type":"ContainerDied","Data":"de16537fc893eed4915059f9157c7ab3f1f41687e445ccf8520c97e29449e7e5"}
Jul 06 22:50:24.224187 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:50:24.224023    2563 scope.go:117] "RemoveContainer" containerID="e7461c6dbbafe0e607f817884660b3a4130585aea8cbb1e6d0dc1b8ee048d9ab"
Jul 06 22:50:24.233528 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:50:24.233506    2563 scope.go:117] "RemoveContainer" containerID="e7461c6dbbafe0e607f817884660b3a4130585aea8cbb1e6d0dc1b8ee048d9ab"
Jul 06 22:50:24.233798 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:50:24.233779    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e7461c6dbbafe0e607f817884660b3a4130585aea8cbb1e6d0dc1b8ee048d9ab\": container with ID starting with e7461c6dbbafe0e607f817884660b3a4130585aea8cbb1e6d0dc1b8ee048d9ab not found: ID does not exist" containerID="e7461c6dbbafe0e607f817884660b3a4130585aea8cbb1e6d0dc1b8ee048d9ab"
Jul 06 22:50:24.233855 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:50:24.233811    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e7461c6dbbafe0e607f817884660b3a4130585aea8cbb1e6d0dc1b8ee048d9ab"} err="failed to get container status \"e7461c6dbbafe0e607f817884660b3a4130585aea8cbb1e6d0dc1b8ee048d9ab\": rpc error: code = NotFound desc = could not find container \"e7461c6dbbafe0e607f817884660b3a4130585aea8cbb1e6d0dc1b8ee048d9ab\": container with ID starting with e7461c6dbbafe0e607f817884660b3a4130585aea8cbb1e6d0dc1b8ee048d9ab not found: ID does not exist"
Jul 06 22:50:24.253569 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:50:24.253543    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-ba18ff93-5ff4544976-zrlbp"]
Jul 06 22:50:24.257539 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:50:24.257517    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-ait-ba18ff93-5ff4544976-zrlbp"]
Jul 06 22:50:25.080219 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:50:25.080187    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f5b2ae39-43d8-4773-b1e9-e9734ac6ca0c" path="/var/lib/kubelet/pods/f5b2ae39-43d8-4773-b1e9-e9734ac6ca0c/volumes"
Jul 06 22:50:30.632079 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:50:30.632033    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-ait-6342cf76-6bd7c7d455-z9brg"]
Jul 06 22:50:30.632465 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:50:30.632446    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f5b2ae39-43d8-4773-b1e9-e9734ac6ca0c" containerName="maas-api"
Jul 06 22:50:30.632465 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:50:30.632464    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="f5b2ae39-43d8-4773-b1e9-e9734ac6ca0c" containerName="maas-api"
Jul 06 22:50:30.632570 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:50:30.632526    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="f5b2ae39-43d8-4773-b1e9-e9734ac6ca0c" containerName="maas-api"
Jul 06 22:50:30.635703 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:50:30.635676    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-6342cf76-6bd7c7d455-z9brg"
Jul 06 22:50:30.638377 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:50:30.638353    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-ait-6342cf76-serving-cert\""
Jul 06 22:50:30.638497 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:50:30.638355    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-f7f72\""
Jul 06 22:50:30.646673 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:50:30.646632    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-6342cf76-6bd7c7d455-z9brg"]
Jul 06 22:50:30.690376 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:50:30.690345    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/871e36a4-511b-44a5-8ecc-6785e26f885f-openshift-service-ca\") pod \"maas-api-e2e-ait-6342cf76-6bd7c7d455-z9brg\" (UID: \"871e36a4-511b-44a5-8ecc-6785e26f885f\") " pod="opendatahub/maas-api-e2e-ait-6342cf76-6bd7c7d455-z9brg"
Jul 06 22:50:30.690539 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:50:30.690397    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pzxtz\" (UniqueName: \"kubernetes.io/projected/871e36a4-511b-44a5-8ecc-6785e26f885f-kube-api-access-pzxtz\") pod \"maas-api-e2e-ait-6342cf76-6bd7c7d455-z9brg\" (UID: \"871e36a4-511b-44a5-8ecc-6785e26f885f\") " pod="opendatahub/maas-api-e2e-ait-6342cf76-6bd7c7d455-z9brg"
Jul 06 22:50:30.690539 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:50:30.690458    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/871e36a4-511b-44a5-8ecc-6785e26f885f-maas-api-tls\") pod \"maas-api-e2e-ait-6342cf76-6bd7c7d455-z9brg\" (UID: \"871e36a4-511b-44a5-8ecc-6785e26f885f\") " pod="opendatahub/maas-api-e2e-ait-6342cf76-6bd7c7d455-z9brg"
Jul 06 22:50:30.791767 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:50:30.791737    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/871e36a4-511b-44a5-8ecc-6785e26f885f-openshift-service-ca\") pod \"maas-api-e2e-ait-6342cf76-6bd7c7d455-z9brg\" (UID: \"871e36a4-511b-44a5-8ecc-6785e26f885f\") " pod="opendatahub/maas-api-e2e-ait-6342cf76-6bd7c7d455-z9brg"
Jul 06 22:50:30.791973 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:50:30.791798    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pzxtz\" (UniqueName: \"kubernetes.io/projected/871e36a4-511b-44a5-8ecc-6785e26f885f-kube-api-access-pzxtz\") pod \"maas-api-e2e-ait-6342cf76-6bd7c7d455-z9brg\" (UID: \"871e36a4-511b-44a5-8ecc-6785e26f885f\") " pod="opendatahub/maas-api-e2e-ait-6342cf76-6bd7c7d455-z9brg"
Jul 06 22:50:30.791973 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:50:30.791833    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/871e36a4-511b-44a5-8ecc-6785e26f885f-maas-api-tls\") pod \"maas-api-e2e-ait-6342cf76-6bd7c7d455-z9brg\" (UID: \"871e36a4-511b-44a5-8ecc-6785e26f885f\") " pod="opendatahub/maas-api-e2e-ait-6342cf76-6bd7c7d455-z9brg"
Jul 06 22:50:30.791973 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:50:30.791942    2563 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-ait-6342cf76-serving-cert: secret "maas-api-e2e-ait-6342cf76-serving-cert" not found
Jul 06 22:50:30.792152 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:50:30.792007    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/871e36a4-511b-44a5-8ecc-6785e26f885f-maas-api-tls podName:871e36a4-511b-44a5-8ecc-6785e26f885f nodeName:}" failed. No retries permitted until 2026-07-06 22:50:31.291990788 +0000 UTC m=+1596.717037308 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/871e36a4-511b-44a5-8ecc-6785e26f885f-maas-api-tls") pod "maas-api-e2e-ait-6342cf76-6bd7c7d455-z9brg" (UID: "871e36a4-511b-44a5-8ecc-6785e26f885f") : secret "maas-api-e2e-ait-6342cf76-serving-cert" not found
Jul 06 22:50:30.792537 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:50:30.792511    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/871e36a4-511b-44a5-8ecc-6785e26f885f-openshift-service-ca\") pod \"maas-api-e2e-ait-6342cf76-6bd7c7d455-z9brg\" (UID: \"871e36a4-511b-44a5-8ecc-6785e26f885f\") " pod="opendatahub/maas-api-e2e-ait-6342cf76-6bd7c7d455-z9brg"
Jul 06 22:50:30.800015 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:50:30.799992    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pzxtz\" (UniqueName: \"kubernetes.io/projected/871e36a4-511b-44a5-8ecc-6785e26f885f-kube-api-access-pzxtz\") pod \"maas-api-e2e-ait-6342cf76-6bd7c7d455-z9brg\" (UID: \"871e36a4-511b-44a5-8ecc-6785e26f885f\") " pod="opendatahub/maas-api-e2e-ait-6342cf76-6bd7c7d455-z9brg"
Jul 06 22:50:31.295245 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:50:31.295215    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/871e36a4-511b-44a5-8ecc-6785e26f885f-maas-api-tls\") pod \"maas-api-e2e-ait-6342cf76-6bd7c7d455-z9brg\" (UID: \"871e36a4-511b-44a5-8ecc-6785e26f885f\") " pod="opendatahub/maas-api-e2e-ait-6342cf76-6bd7c7d455-z9brg"
Jul 06 22:50:31.297577 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:50:31.297550    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/871e36a4-511b-44a5-8ecc-6785e26f885f-maas-api-tls\") pod \"maas-api-e2e-ait-6342cf76-6bd7c7d455-z9brg\" (UID: \"871e36a4-511b-44a5-8ecc-6785e26f885f\") " pod="opendatahub/maas-api-e2e-ait-6342cf76-6bd7c7d455-z9brg"
Jul 06 22:50:31.548395 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:50:31.548305    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-6342cf76-6bd7c7d455-z9brg"
Jul 06 22:50:31.687826 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:50:31.687797    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-6342cf76-6bd7c7d455-z9brg"]
Jul 06 22:50:31.689708 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:50:31.689680    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod871e36a4_511b_44a5_8ecc_6785e26f885f.slice/crio-3ad4d9ee005f1fc8b4597520150c3d32550b2ef86d4196f629103d31da327dc7 WatchSource:0}: Error finding container 3ad4d9ee005f1fc8b4597520150c3d32550b2ef86d4196f629103d31da327dc7: Status 404 returned error can't find the container with id 3ad4d9ee005f1fc8b4597520150c3d32550b2ef86d4196f629103d31da327dc7
Jul 06 22:50:32.262272 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:50:32.262238    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-6342cf76-6bd7c7d455-z9brg" event={"ID":"871e36a4-511b-44a5-8ecc-6785e26f885f","Type":"ContainerStarted","Data":"0913cb7fca66f66e21fa36b21581d6274d232d9fe2899aa1bf19282634f43d52"}
Jul 06 22:50:32.262438 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:50:32.262286    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-6342cf76-6bd7c7d455-z9brg" event={"ID":"871e36a4-511b-44a5-8ecc-6785e26f885f","Type":"ContainerStarted","Data":"3ad4d9ee005f1fc8b4597520150c3d32550b2ef86d4196f629103d31da327dc7"}
Jul 06 22:50:32.262438 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:50:32.262355    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-ait-6342cf76-6bd7c7d455-z9brg"
Jul 06 22:50:32.279968 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:50:32.279921    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-ait-6342cf76-6bd7c7d455-z9brg" podStartSLOduration=1.9367715840000002 podStartE2EDuration="2.279907131s" totalImagesPullingTime="343.135547ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:50:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:50:32.27825622 +0000 UTC m=+1597.703302762" watchObservedRunningTime="2026-07-06 22:50:32.279907131 +0000 UTC m=+1597.704953672"
Jul 06 22:50:33.174769 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:50:33.174730    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9q89m"]
Jul 06 22:50:36.014568 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:50:36.014535    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-6342cf76-6bd7c7d455-z9brg"]
Jul 06 22:50:36.015058 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:50:36.014744    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-ait-6342cf76-6bd7c7d455-z9brg" podUID="871e36a4-511b-44a5-8ecc-6785e26f885f" containerName="maas-api" containerID="cri-o://0913cb7fca66f66e21fa36b21581d6274d232d9fe2899aa1bf19282634f43d52" gracePeriod=30
Jul 06 22:50:36.265534 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:50:36.265475    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-6342cf76-6bd7c7d455-z9brg"
Jul 06 22:50:36.282341 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:50:36.282309    2563 generic.go:356] "Generic (PLEG): container finished" podID="871e36a4-511b-44a5-8ecc-6785e26f885f" containerID="0913cb7fca66f66e21fa36b21581d6274d232d9fe2899aa1bf19282634f43d52" exitCode=0
Jul 06 22:50:36.282482 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:50:36.282385    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-6342cf76-6bd7c7d455-z9brg" event={"ID":"871e36a4-511b-44a5-8ecc-6785e26f885f","Type":"ContainerDied","Data":"0913cb7fca66f66e21fa36b21581d6274d232d9fe2899aa1bf19282634f43d52"}
Jul 06 22:50:36.282482 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:50:36.282395    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-6342cf76-6bd7c7d455-z9brg"
Jul 06 22:50:36.282482 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:50:36.282438    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-6342cf76-6bd7c7d455-z9brg" event={"ID":"871e36a4-511b-44a5-8ecc-6785e26f885f","Type":"ContainerDied","Data":"3ad4d9ee005f1fc8b4597520150c3d32550b2ef86d4196f629103d31da327dc7"}
Jul 06 22:50:36.282482 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:50:36.282469    2563 scope.go:117] "RemoveContainer" containerID="0913cb7fca66f66e21fa36b21581d6274d232d9fe2899aa1bf19282634f43d52"
Jul 06 22:50:36.292645 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:50:36.292621    2563 scope.go:117] "RemoveContainer" containerID="0913cb7fca66f66e21fa36b21581d6274d232d9fe2899aa1bf19282634f43d52"
Jul 06 22:50:36.292970 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:50:36.292942    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0913cb7fca66f66e21fa36b21581d6274d232d9fe2899aa1bf19282634f43d52\": container with ID starting with 0913cb7fca66f66e21fa36b21581d6274d232d9fe2899aa1bf19282634f43d52 not found: ID does not exist" containerID="0913cb7fca66f66e21fa36b21581d6274d232d9fe2899aa1bf19282634f43d52"
Jul 06 22:50:36.293059 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:50:36.292981    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0913cb7fca66f66e21fa36b21581d6274d232d9fe2899aa1bf19282634f43d52"} err="failed to get container status \"0913cb7fca66f66e21fa36b21581d6274d232d9fe2899aa1bf19282634f43d52\": rpc error: code = NotFound desc = could not find container \"0913cb7fca66f66e21fa36b21581d6274d232d9fe2899aa1bf19282634f43d52\": container with ID starting with 0913cb7fca66f66e21fa36b21581d6274d232d9fe2899aa1bf19282634f43d52 not found: ID does not exist"
Jul 06 22:50:36.338389 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:50:36.338362    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/871e36a4-511b-44a5-8ecc-6785e26f885f-maas-api-tls\") pod \"871e36a4-511b-44a5-8ecc-6785e26f885f\" (UID: \"871e36a4-511b-44a5-8ecc-6785e26f885f\") "
Jul 06 22:50:36.338557 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:50:36.338406    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pzxtz\" (UniqueName: \"kubernetes.io/projected/871e36a4-511b-44a5-8ecc-6785e26f885f-kube-api-access-pzxtz\") pod \"871e36a4-511b-44a5-8ecc-6785e26f885f\" (UID: \"871e36a4-511b-44a5-8ecc-6785e26f885f\") "
Jul 06 22:50:36.338557 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:50:36.338516    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/871e36a4-511b-44a5-8ecc-6785e26f885f-openshift-service-ca\") pod \"871e36a4-511b-44a5-8ecc-6785e26f885f\" (UID: \"871e36a4-511b-44a5-8ecc-6785e26f885f\") "
Jul 06 22:50:36.338814 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:50:36.338788    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/871e36a4-511b-44a5-8ecc-6785e26f885f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "871e36a4-511b-44a5-8ecc-6785e26f885f" (UID: "871e36a4-511b-44a5-8ecc-6785e26f885f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 22:50:36.340579 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:50:36.340558    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/871e36a4-511b-44a5-8ecc-6785e26f885f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "871e36a4-511b-44a5-8ecc-6785e26f885f" (UID: "871e36a4-511b-44a5-8ecc-6785e26f885f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 22:50:36.340678 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:50:36.340594    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/871e36a4-511b-44a5-8ecc-6785e26f885f-kube-api-access-pzxtz" (OuterVolumeSpecName: "kube-api-access-pzxtz") pod "871e36a4-511b-44a5-8ecc-6785e26f885f" (UID: "871e36a4-511b-44a5-8ecc-6785e26f885f"). InnerVolumeSpecName "kube-api-access-pzxtz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 22:50:36.439623 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:50:36.439591    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/871e36a4-511b-44a5-8ecc-6785e26f885f-maas-api-tls\") on node \"ip-10-0-133-105.ec2.internal\" DevicePath \"\""
Jul 06 22:50:36.439623 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:50:36.439617    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pzxtz\" (UniqueName: \"kubernetes.io/projected/871e36a4-511b-44a5-8ecc-6785e26f885f-kube-api-access-pzxtz\") on node \"ip-10-0-133-105.ec2.internal\" DevicePath \"\""
Jul 06 22:50:36.439623 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:50:36.439628    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/871e36a4-511b-44a5-8ecc-6785e26f885f-openshift-service-ca\") on node \"ip-10-0-133-105.ec2.internal\" DevicePath \"\""
Jul 06 22:50:36.609137 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:50:36.609104    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-6342cf76-6bd7c7d455-z9brg"]
Jul 06 22:50:36.613070 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:50:36.613045    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-ait-6342cf76-6bd7c7d455-z9brg"]
Jul 06 22:50:37.082870 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:50:37.082830    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="871e36a4-511b-44a5-8ecc-6785e26f885f" path="/var/lib/kubelet/pods/871e36a4-511b-44a5-8ecc-6785e26f885f/volumes"
Jul 06 22:50:44.072521 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:50:44.072491    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9q89m"]
Jul 06 22:50:49.239537 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:50:49.239502    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-derive-6cd14b60-7d9b474575-pfwjk"]
Jul 06 22:50:49.240144 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:50:49.240122    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="871e36a4-511b-44a5-8ecc-6785e26f885f" containerName="maas-api"
Jul 06 22:50:49.240144 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:50:49.240146    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="871e36a4-511b-44a5-8ecc-6785e26f885f" containerName="maas-api"
Jul 06 22:50:49.240347 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:50:49.240321    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="871e36a4-511b-44a5-8ecc-6785e26f885f" containerName="maas-api"
Jul 06 22:50:49.246053 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:50:49.246032    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-6cd14b60-7d9b474575-pfwjk"
Jul 06 22:50:49.248431 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:50:49.248402    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-derive-6cd14b60-serving-cert\""
Jul 06 22:50:49.248547 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:50:49.248455    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-f7f72\""
Jul 06 22:50:49.254295 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:50:49.253831    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-derive-6cd14b60-7d9b474575-pfwjk"]
Jul 06 22:50:49.348950 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:50:49.348924    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9ce5e446-56a6-479d-80f2-64c879c3b0a5-maas-api-tls\") pod \"maas-api-e2e-derive-6cd14b60-7d9b474575-pfwjk\" (UID: \"9ce5e446-56a6-479d-80f2-64c879c3b0a5\") " pod="opendatahub/maas-api-e2e-derive-6cd14b60-7d9b474575-pfwjk"
Jul 06 22:50:49.349129 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:50:49.348961    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-56glm\" (UniqueName: \"kubernetes.io/projected/9ce5e446-56a6-479d-80f2-64c879c3b0a5-kube-api-access-56glm\") pod \"maas-api-e2e-derive-6cd14b60-7d9b474575-pfwjk\" (UID: \"9ce5e446-56a6-479d-80f2-64c879c3b0a5\") " pod="opendatahub/maas-api-e2e-derive-6cd14b60-7d9b474575-pfwjk"
Jul 06 22:50:49.349129 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:50:49.349058    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ce5e446-56a6-479d-80f2-64c879c3b0a5-openshift-service-ca\") pod \"maas-api-e2e-derive-6cd14b60-7d9b474575-pfwjk\" (UID: \"9ce5e446-56a6-479d-80f2-64c879c3b0a5\") " pod="opendatahub/maas-api-e2e-derive-6cd14b60-7d9b474575-pfwjk"
Jul 06 22:50:49.449572 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:50:49.449533    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9ce5e446-56a6-479d-80f2-64c879c3b0a5-maas-api-tls\") pod \"maas-api-e2e-derive-6cd14b60-7d9b474575-pfwjk\" (UID: \"9ce5e446-56a6-479d-80f2-64c879c3b0a5\") " pod="opendatahub/maas-api-e2e-derive-6cd14b60-7d9b474575-pfwjk"
Jul 06 22:50:49.449769 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:50:49.449587    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-56glm\" (UniqueName: \"kubernetes.io/projected/9ce5e446-56a6-479d-80f2-64c879c3b0a5-kube-api-access-56glm\") pod \"maas-api-e2e-derive-6cd14b60-7d9b474575-pfwjk\" (UID: \"9ce5e446-56a6-479d-80f2-64c879c3b0a5\") " pod="opendatahub/maas-api-e2e-derive-6cd14b60-7d9b474575-pfwjk"
Jul 06 22:50:49.449769 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:50:49.449660    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ce5e446-56a6-479d-80f2-64c879c3b0a5-openshift-service-ca\") pod \"maas-api-e2e-derive-6cd14b60-7d9b474575-pfwjk\" (UID: \"9ce5e446-56a6-479d-80f2-64c879c3b0a5\") " pod="opendatahub/maas-api-e2e-derive-6cd14b60-7d9b474575-pfwjk"
Jul 06 22:50:49.450458 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:50:49.450377    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ce5e446-56a6-479d-80f2-64c879c3b0a5-openshift-service-ca\") pod \"maas-api-e2e-derive-6cd14b60-7d9b474575-pfwjk\" (UID: \"9ce5e446-56a6-479d-80f2-64c879c3b0a5\") " pod="opendatahub/maas-api-e2e-derive-6cd14b60-7d9b474575-pfwjk"
Jul 06 22:50:49.452235 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:50:49.452214    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9ce5e446-56a6-479d-80f2-64c879c3b0a5-maas-api-tls\") pod \"maas-api-e2e-derive-6cd14b60-7d9b474575-pfwjk\" (UID: \"9ce5e446-56a6-479d-80f2-64c879c3b0a5\") " pod="opendatahub/maas-api-e2e-derive-6cd14b60-7d9b474575-pfwjk"
Jul 06 22:50:49.458462 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:50:49.458437    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-56glm\" (UniqueName: \"kubernetes.io/projected/9ce5e446-56a6-479d-80f2-64c879c3b0a5-kube-api-access-56glm\") pod \"maas-api-e2e-derive-6cd14b60-7d9b474575-pfwjk\" (UID: \"9ce5e446-56a6-479d-80f2-64c879c3b0a5\") " pod="opendatahub/maas-api-e2e-derive-6cd14b60-7d9b474575-pfwjk"
Jul 06 22:50:49.559091 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:50:49.559015    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-6cd14b60-7d9b474575-pfwjk"
Jul 06 22:50:49.701624 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:50:49.701600    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-derive-6cd14b60-7d9b474575-pfwjk"]
Jul 06 22:50:49.705012 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:50:49.704985    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ce5e446_56a6_479d_80f2_64c879c3b0a5.slice/crio-7e9c5da455fdcc318e05f269e97808102aa2f1e2b891c948fe7c0ea20e5bb703 WatchSource:0}: Error finding container 7e9c5da455fdcc318e05f269e97808102aa2f1e2b891c948fe7c0ea20e5bb703: Status 404 returned error can't find the container with id 7e9c5da455fdcc318e05f269e97808102aa2f1e2b891c948fe7c0ea20e5bb703
Jul 06 22:50:50.354745 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:50:50.354710    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-6cd14b60-7d9b474575-pfwjk" event={"ID":"9ce5e446-56a6-479d-80f2-64c879c3b0a5","Type":"ContainerStarted","Data":"4a16aa030fa179b3a27e7fbef361063707f887c460e1b9f103144a2a0e44a7a3"}
Jul 06 22:50:50.355137 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:50:50.354770    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-derive-6cd14b60-7d9b474575-pfwjk"
Jul 06 22:50:50.355137 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:50:50.354795    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-6cd14b60-7d9b474575-pfwjk" event={"ID":"9ce5e446-56a6-479d-80f2-64c879c3b0a5","Type":"ContainerStarted","Data":"7e9c5da455fdcc318e05f269e97808102aa2f1e2b891c948fe7c0ea20e5bb703"}
Jul 06 22:50:50.371991 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:50:50.371861    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-derive-6cd14b60-7d9b474575-pfwjk" podStartSLOduration=0.962771971 podStartE2EDuration="1.371842394s" totalImagesPullingTime="409.070423ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:50:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:50:50.371017665 +0000 UTC m=+1615.796064198" watchObservedRunningTime="2026-07-06 22:50:50.371842394 +0000 UTC m=+1615.796888936"
Jul 06 22:50:53.876475 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:50:53.876439    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-derive-6cd14b60-7d9b474575-pfwjk"]
Jul 06 22:50:53.876862 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:50:53.876709    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-derive-6cd14b60-7d9b474575-pfwjk" podUID="9ce5e446-56a6-479d-80f2-64c879c3b0a5" containerName="maas-api" containerID="cri-o://4a16aa030fa179b3a27e7fbef361063707f887c460e1b9f103144a2a0e44a7a3" gracePeriod=30
Jul 06 22:50:54.135893 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:50:54.135822    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-6cd14b60-7d9b474575-pfwjk"
Jul 06 22:50:54.188888 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:50:54.188858    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ce5e446-56a6-479d-80f2-64c879c3b0a5-openshift-service-ca\") pod \"9ce5e446-56a6-479d-80f2-64c879c3b0a5\" (UID: \"9ce5e446-56a6-479d-80f2-64c879c3b0a5\") "
Jul 06 22:50:54.189059 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:50:54.188943    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9ce5e446-56a6-479d-80f2-64c879c3b0a5-maas-api-tls\") pod \"9ce5e446-56a6-479d-80f2-64c879c3b0a5\" (UID: \"9ce5e446-56a6-479d-80f2-64c879c3b0a5\") "
Jul 06 22:50:54.189059 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:50:54.188992    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-56glm\" (UniqueName: \"kubernetes.io/projected/9ce5e446-56a6-479d-80f2-64c879c3b0a5-kube-api-access-56glm\") pod \"9ce5e446-56a6-479d-80f2-64c879c3b0a5\" (UID: \"9ce5e446-56a6-479d-80f2-64c879c3b0a5\") "
Jul 06 22:50:54.189264 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:50:54.189233    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9ce5e446-56a6-479d-80f2-64c879c3b0a5-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "9ce5e446-56a6-479d-80f2-64c879c3b0a5" (UID: "9ce5e446-56a6-479d-80f2-64c879c3b0a5"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 22:50:54.191092 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:50:54.191068    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9ce5e446-56a6-479d-80f2-64c879c3b0a5-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "9ce5e446-56a6-479d-80f2-64c879c3b0a5" (UID: "9ce5e446-56a6-479d-80f2-64c879c3b0a5"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 22:50:54.191214 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:50:54.191119    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ce5e446-56a6-479d-80f2-64c879c3b0a5-kube-api-access-56glm" (OuterVolumeSpecName: "kube-api-access-56glm") pod "9ce5e446-56a6-479d-80f2-64c879c3b0a5" (UID: "9ce5e446-56a6-479d-80f2-64c879c3b0a5"). InnerVolumeSpecName "kube-api-access-56glm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 22:50:54.290407 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:50:54.290373    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-56glm\" (UniqueName: \"kubernetes.io/projected/9ce5e446-56a6-479d-80f2-64c879c3b0a5-kube-api-access-56glm\") on node \"ip-10-0-133-105.ec2.internal\" DevicePath \"\""
Jul 06 22:50:54.290407 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:50:54.290403    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ce5e446-56a6-479d-80f2-64c879c3b0a5-openshift-service-ca\") on node \"ip-10-0-133-105.ec2.internal\" DevicePath \"\""
Jul 06 22:50:54.290407 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:50:54.290414    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9ce5e446-56a6-479d-80f2-64c879c3b0a5-maas-api-tls\") on node \"ip-10-0-133-105.ec2.internal\" DevicePath \"\""
Jul 06 22:50:54.378654 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:50:54.378620    2563 generic.go:356] "Generic (PLEG): container finished" podID="9ce5e446-56a6-479d-80f2-64c879c3b0a5" containerID="4a16aa030fa179b3a27e7fbef361063707f887c460e1b9f103144a2a0e44a7a3" exitCode=0
Jul 06 22:50:54.378816 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:50:54.378679    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-6cd14b60-7d9b474575-pfwjk" event={"ID":"9ce5e446-56a6-479d-80f2-64c879c3b0a5","Type":"ContainerDied","Data":"4a16aa030fa179b3a27e7fbef361063707f887c460e1b9f103144a2a0e44a7a3"}
Jul 06 22:50:54.378816 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:50:54.378684    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-6cd14b60-7d9b474575-pfwjk"
Jul 06 22:50:54.378816 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:50:54.378721    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-6cd14b60-7d9b474575-pfwjk" event={"ID":"9ce5e446-56a6-479d-80f2-64c879c3b0a5","Type":"ContainerDied","Data":"7e9c5da455fdcc318e05f269e97808102aa2f1e2b891c948fe7c0ea20e5bb703"}
Jul 06 22:50:54.378816 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:50:54.378744    2563 scope.go:117] "RemoveContainer" containerID="4a16aa030fa179b3a27e7fbef361063707f887c460e1b9f103144a2a0e44a7a3"
Jul 06 22:50:54.388427 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:50:54.388406    2563 scope.go:117] "RemoveContainer" containerID="4a16aa030fa179b3a27e7fbef361063707f887c460e1b9f103144a2a0e44a7a3"
Jul 06 22:50:54.388704 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:50:54.388684    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4a16aa030fa179b3a27e7fbef361063707f887c460e1b9f103144a2a0e44a7a3\": container with ID starting with 4a16aa030fa179b3a27e7fbef361063707f887c460e1b9f103144a2a0e44a7a3 not found: ID does not exist" containerID="4a16aa030fa179b3a27e7fbef361063707f887c460e1b9f103144a2a0e44a7a3"
Jul 06 22:50:54.388781 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:50:54.388715    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4a16aa030fa179b3a27e7fbef361063707f887c460e1b9f103144a2a0e44a7a3"} err="failed to get container status \"4a16aa030fa179b3a27e7fbef361063707f887c460e1b9f103144a2a0e44a7a3\": rpc error: code = NotFound desc = could not find container \"4a16aa030fa179b3a27e7fbef361063707f887c460e1b9f103144a2a0e44a7a3\": container with ID starting with 4a16aa030fa179b3a27e7fbef361063707f887c460e1b9f103144a2a0e44a7a3 not found: ID does not exist"
Jul 06 22:50:54.400478 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:50:54.400451    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-derive-6cd14b60-7d9b474575-pfwjk"]
Jul 06 22:50:54.410107 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:50:54.410087    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-derive-6cd14b60-7d9b474575-pfwjk"]
Jul 06 22:50:55.080824 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:50:55.080745    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9ce5e446-56a6-479d-80f2-64c879c3b0a5" path="/var/lib/kubelet/pods/9ce5e446-56a6-479d-80f2-64c879c3b0a5/volumes"
Jul 06 22:52:22.877706 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:52:22.877614    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9q89m"]
Jul 06 22:52:30.378434 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:52:30.378397    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9q89m"]
Jul 06 22:52:39.067572 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:52:39.067536    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9q89m"]
Jul 06 22:52:39.412666 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:52:39.412593    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-mt-7d7e43b3-8f4dc9598-mw5sn"]
Jul 06 22:52:39.412983 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:52:39.412971    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ce5e446-56a6-479d-80f2-64c879c3b0a5" containerName="maas-api"
Jul 06 22:52:39.413026 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:52:39.412985    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ce5e446-56a6-479d-80f2-64c879c3b0a5" containerName="maas-api"
Jul 06 22:52:39.413064 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:52:39.413039    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="9ce5e446-56a6-479d-80f2-64c879c3b0a5" containerName="maas-api"
Jul 06 22:52:39.415427 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:52:39.415399    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-7d7e43b3-8f4dc9598-mw5sn"
Jul 06 22:52:39.418026 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:52:39.418002    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-f7f72\""
Jul 06 22:52:39.418026 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:52:39.418015    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-mt-7d7e43b3-serving-cert\""
Jul 06 22:52:39.426814 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:52:39.426794    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-7d7e43b3-8f4dc9598-mw5sn"]
Jul 06 22:52:39.526289 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:52:39.526257    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/540fe426-9e18-4f30-9067-72ab9899933d-openshift-service-ca\") pod \"maas-api-e2e-mt-7d7e43b3-8f4dc9598-mw5sn\" (UID: \"540fe426-9e18-4f30-9067-72ab9899933d\") " pod="opendatahub/maas-api-e2e-mt-7d7e43b3-8f4dc9598-mw5sn"
Jul 06 22:52:39.526289 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:52:39.526301    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dg89m\" (UniqueName: \"kubernetes.io/projected/540fe426-9e18-4f30-9067-72ab9899933d-kube-api-access-dg89m\") pod \"maas-api-e2e-mt-7d7e43b3-8f4dc9598-mw5sn\" (UID: \"540fe426-9e18-4f30-9067-72ab9899933d\") " pod="opendatahub/maas-api-e2e-mt-7d7e43b3-8f4dc9598-mw5sn"
Jul 06 22:52:39.526487 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:52:39.526383    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/540fe426-9e18-4f30-9067-72ab9899933d-maas-api-tls\") pod \"maas-api-e2e-mt-7d7e43b3-8f4dc9598-mw5sn\" (UID: \"540fe426-9e18-4f30-9067-72ab9899933d\") " pod="opendatahub/maas-api-e2e-mt-7d7e43b3-8f4dc9598-mw5sn"
Jul 06 22:52:39.627021 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:52:39.626988    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/540fe426-9e18-4f30-9067-72ab9899933d-maas-api-tls\") pod \"maas-api-e2e-mt-7d7e43b3-8f4dc9598-mw5sn\" (UID: \"540fe426-9e18-4f30-9067-72ab9899933d\") " pod="opendatahub/maas-api-e2e-mt-7d7e43b3-8f4dc9598-mw5sn"
Jul 06 22:52:39.627239 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:52:39.627049    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/540fe426-9e18-4f30-9067-72ab9899933d-openshift-service-ca\") pod \"maas-api-e2e-mt-7d7e43b3-8f4dc9598-mw5sn\" (UID: \"540fe426-9e18-4f30-9067-72ab9899933d\") " pod="opendatahub/maas-api-e2e-mt-7d7e43b3-8f4dc9598-mw5sn"
Jul 06 22:52:39.627239 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:52:39.627078    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dg89m\" (UniqueName: \"kubernetes.io/projected/540fe426-9e18-4f30-9067-72ab9899933d-kube-api-access-dg89m\") pod \"maas-api-e2e-mt-7d7e43b3-8f4dc9598-mw5sn\" (UID: \"540fe426-9e18-4f30-9067-72ab9899933d\") " pod="opendatahub/maas-api-e2e-mt-7d7e43b3-8f4dc9598-mw5sn"
Jul 06 22:52:39.627779 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:52:39.627762    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/540fe426-9e18-4f30-9067-72ab9899933d-openshift-service-ca\") pod \"maas-api-e2e-mt-7d7e43b3-8f4dc9598-mw5sn\" (UID: \"540fe426-9e18-4f30-9067-72ab9899933d\") " pod="opendatahub/maas-api-e2e-mt-7d7e43b3-8f4dc9598-mw5sn"
Jul 06 22:52:39.629627 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:52:39.629606    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/540fe426-9e18-4f30-9067-72ab9899933d-maas-api-tls\") pod \"maas-api-e2e-mt-7d7e43b3-8f4dc9598-mw5sn\" (UID: \"540fe426-9e18-4f30-9067-72ab9899933d\") " pod="opendatahub/maas-api-e2e-mt-7d7e43b3-8f4dc9598-mw5sn"
Jul 06 22:52:39.635514 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:52:39.635491    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dg89m\" (UniqueName: \"kubernetes.io/projected/540fe426-9e18-4f30-9067-72ab9899933d-kube-api-access-dg89m\") pod \"maas-api-e2e-mt-7d7e43b3-8f4dc9598-mw5sn\" (UID: \"540fe426-9e18-4f30-9067-72ab9899933d\") " pod="opendatahub/maas-api-e2e-mt-7d7e43b3-8f4dc9598-mw5sn"
Jul 06 22:52:39.728278 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:52:39.728248    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-7d7e43b3-8f4dc9598-mw5sn"
Jul 06 22:52:39.869800 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:52:39.869777    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-7d7e43b3-8f4dc9598-mw5sn"]
Jul 06 22:52:39.871720 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:52:39.871693    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod540fe426_9e18_4f30_9067_72ab9899933d.slice/crio-8580c07b72187f50a3c86997849f46d65855cf1933471080e248f0b73f088cb0 WatchSource:0}: Error finding container 8580c07b72187f50a3c86997849f46d65855cf1933471080e248f0b73f088cb0: Status 404 returned error can't find the container with id 8580c07b72187f50a3c86997849f46d65855cf1933471080e248f0b73f088cb0
Jul 06 22:52:40.869595 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:52:40.869508    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-7d7e43b3-8f4dc9598-mw5sn" event={"ID":"540fe426-9e18-4f30-9067-72ab9899933d","Type":"ContainerStarted","Data":"099676de5a9f8dc01a0ba01ee571fc5fe10acf9421d1f306c78f70029dc8fce6"}
Jul 06 22:52:40.869595 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:52:40.869576    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-7d7e43b3-8f4dc9598-mw5sn" event={"ID":"540fe426-9e18-4f30-9067-72ab9899933d","Type":"ContainerStarted","Data":"8580c07b72187f50a3c86997849f46d65855cf1933471080e248f0b73f088cb0"}
Jul 06 22:52:40.869969 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:52:40.869650    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-mt-7d7e43b3-8f4dc9598-mw5sn"
Jul 06 22:52:40.889035 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:52:40.888988    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-mt-7d7e43b3-8f4dc9598-mw5sn" podStartSLOduration=1.309945153 podStartE2EDuration="1.888975492s" totalImagesPullingTime="579.030339ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:52:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:52:40.88726401 +0000 UTC m=+1726.312310551" watchObservedRunningTime="2026-07-06 22:52:40.888975492 +0000 UTC m=+1726.314022391"
Jul 06 22:52:46.884742 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:52:46.884711    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-mt-7d7e43b3-8f4dc9598-mw5sn"
Jul 06 22:52:49.168107 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:52:49.168072    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9q89m"]
Jul 06 22:52:51.629792 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:52:51.629760    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-7d7e43b3-8f4dc9598-mw5sn"]
Jul 06 22:52:51.630298 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:52:51.630010    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-mt-7d7e43b3-8f4dc9598-mw5sn" podUID="540fe426-9e18-4f30-9067-72ab9899933d" containerName="maas-api" containerID="cri-o://099676de5a9f8dc01a0ba01ee571fc5fe10acf9421d1f306c78f70029dc8fce6" gracePeriod=30
Jul 06 22:52:51.886228 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:52:51.886138    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-7d7e43b3-8f4dc9598-mw5sn"
Jul 06 22:52:51.926583 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:52:51.926545    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dg89m\" (UniqueName: \"kubernetes.io/projected/540fe426-9e18-4f30-9067-72ab9899933d-kube-api-access-dg89m\") pod \"540fe426-9e18-4f30-9067-72ab9899933d\" (UID: \"540fe426-9e18-4f30-9067-72ab9899933d\") "
Jul 06 22:52:51.926755 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:52:51.926655    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/540fe426-9e18-4f30-9067-72ab9899933d-openshift-service-ca\") pod \"540fe426-9e18-4f30-9067-72ab9899933d\" (UID: \"540fe426-9e18-4f30-9067-72ab9899933d\") "
Jul 06 22:52:51.926755 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:52:51.926682    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/540fe426-9e18-4f30-9067-72ab9899933d-maas-api-tls\") pod \"540fe426-9e18-4f30-9067-72ab9899933d\" (UID: \"540fe426-9e18-4f30-9067-72ab9899933d\") "
Jul 06 22:52:51.927049 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:52:51.927022    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/540fe426-9e18-4f30-9067-72ab9899933d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "540fe426-9e18-4f30-9067-72ab9899933d" (UID: "540fe426-9e18-4f30-9067-72ab9899933d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 22:52:51.928978 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:52:51.928949    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/540fe426-9e18-4f30-9067-72ab9899933d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "540fe426-9e18-4f30-9067-72ab9899933d" (UID: "540fe426-9e18-4f30-9067-72ab9899933d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 22:52:51.929295 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:52:51.929272    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/540fe426-9e18-4f30-9067-72ab9899933d-kube-api-access-dg89m" (OuterVolumeSpecName: "kube-api-access-dg89m") pod "540fe426-9e18-4f30-9067-72ab9899933d" (UID: "540fe426-9e18-4f30-9067-72ab9899933d"). InnerVolumeSpecName "kube-api-access-dg89m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 22:52:51.929716 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:52:51.929684    2563 generic.go:356] "Generic (PLEG): container finished" podID="540fe426-9e18-4f30-9067-72ab9899933d" containerID="099676de5a9f8dc01a0ba01ee571fc5fe10acf9421d1f306c78f70029dc8fce6" exitCode=0
Jul 06 22:52:51.929815 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:52:51.929766    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-7d7e43b3-8f4dc9598-mw5sn" event={"ID":"540fe426-9e18-4f30-9067-72ab9899933d","Type":"ContainerDied","Data":"099676de5a9f8dc01a0ba01ee571fc5fe10acf9421d1f306c78f70029dc8fce6"}
Jul 06 22:52:51.929815 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:52:51.929783    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-7d7e43b3-8f4dc9598-mw5sn"
Jul 06 22:52:51.929928 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:52:51.929823    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-7d7e43b3-8f4dc9598-mw5sn" event={"ID":"540fe426-9e18-4f30-9067-72ab9899933d","Type":"ContainerDied","Data":"8580c07b72187f50a3c86997849f46d65855cf1933471080e248f0b73f088cb0"}
Jul 06 22:52:51.929928 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:52:51.929855    2563 scope.go:117] "RemoveContainer" containerID="099676de5a9f8dc01a0ba01ee571fc5fe10acf9421d1f306c78f70029dc8fce6"
Jul 06 22:52:51.945678 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:52:51.945660    2563 scope.go:117] "RemoveContainer" containerID="099676de5a9f8dc01a0ba01ee571fc5fe10acf9421d1f306c78f70029dc8fce6"
Jul 06 22:52:51.945966 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:52:51.945949    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"099676de5a9f8dc01a0ba01ee571fc5fe10acf9421d1f306c78f70029dc8fce6\": container with ID starting with 099676de5a9f8dc01a0ba01ee571fc5fe10acf9421d1f306c78f70029dc8fce6 not found: ID does not exist" containerID="099676de5a9f8dc01a0ba01ee571fc5fe10acf9421d1f306c78f70029dc8fce6"
Jul 06 22:52:51.946005 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:52:51.945975    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"099676de5a9f8dc01a0ba01ee571fc5fe10acf9421d1f306c78f70029dc8fce6"} err="failed to get container status \"099676de5a9f8dc01a0ba01ee571fc5fe10acf9421d1f306c78f70029dc8fce6\": rpc error: code = NotFound desc = could not find container \"099676de5a9f8dc01a0ba01ee571fc5fe10acf9421d1f306c78f70029dc8fce6\": container with ID starting with 099676de5a9f8dc01a0ba01ee571fc5fe10acf9421d1f306c78f70029dc8fce6 not found: ID does not exist"
Jul 06 22:52:51.956093 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:52:51.956067    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-7d7e43b3-8f4dc9598-mw5sn"]
Jul 06 22:52:51.959858 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:52:51.959836    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-mt-7d7e43b3-8f4dc9598-mw5sn"]
Jul 06 22:52:52.028200 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:52:52.028146    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/540fe426-9e18-4f30-9067-72ab9899933d-openshift-service-ca\") on node \"ip-10-0-133-105.ec2.internal\" DevicePath \"\""
Jul 06 22:52:52.028200 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:52:52.028202    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/540fe426-9e18-4f30-9067-72ab9899933d-maas-api-tls\") on node \"ip-10-0-133-105.ec2.internal\" DevicePath \"\""
Jul 06 22:52:52.028377 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:52:52.028220    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dg89m\" (UniqueName: \"kubernetes.io/projected/540fe426-9e18-4f30-9067-72ab9899933d-kube-api-access-dg89m\") on node \"ip-10-0-133-105.ec2.internal\" DevicePath \"\""
Jul 06 22:52:53.079827 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:52:53.079795    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="540fe426-9e18-4f30-9067-72ab9899933d" path="/var/lib/kubelet/pods/540fe426-9e18-4f30-9067-72ab9899933d/volumes"
Jul 06 22:52:54.880683 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:52:54.880642    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="opendatahub/maas-api-e2e-mt-7d7e43b3-8f4dc9598-mw5sn" podUID="540fe426-9e18-4f30-9067-72ab9899933d" containerName="maas-api" probeResult="failure" output="Get \"https://10.133.0.38:8443/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Jul 06 22:52:59.775439 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:52:59.775405    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9q89m"]
Jul 06 22:53:20.613282 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:53:20.613250    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-mt-75c546e0-85b97d64d-8c64l"]
Jul 06 22:53:20.613629 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:53:20.613602    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="540fe426-9e18-4f30-9067-72ab9899933d" containerName="maas-api"
Jul 06 22:53:20.613629 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:53:20.613613    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="540fe426-9e18-4f30-9067-72ab9899933d" containerName="maas-api"
Jul 06 22:53:20.613700 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:53:20.613680    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="540fe426-9e18-4f30-9067-72ab9899933d" containerName="maas-api"
Jul 06 22:53:20.621779 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:53:20.621752    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-75c546e0-85b97d64d-8c64l"
Jul 06 22:53:20.624895 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:53:20.624872    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-f7f72\""
Jul 06 22:53:20.625025 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:53:20.624874    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-mt-75c546e0-serving-cert\""
Jul 06 22:53:20.631326 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:53:20.631285    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-75c546e0-85b97d64d-8c64l"]
Jul 06 22:53:20.782649 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:53:20.782612    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d4b8a3d9-c010-4a46-9f70-d9e5208a75e7-openshift-service-ca\") pod \"maas-api-e2e-mt-75c546e0-85b97d64d-8c64l\" (UID: \"d4b8a3d9-c010-4a46-9f70-d9e5208a75e7\") " pod="opendatahub/maas-api-e2e-mt-75c546e0-85b97d64d-8c64l"
Jul 06 22:53:20.782649 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:53:20.782647    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d4b8a3d9-c010-4a46-9f70-d9e5208a75e7-maas-api-tls\") pod \"maas-api-e2e-mt-75c546e0-85b97d64d-8c64l\" (UID: \"d4b8a3d9-c010-4a46-9f70-d9e5208a75e7\") " pod="opendatahub/maas-api-e2e-mt-75c546e0-85b97d64d-8c64l"
Jul 06 22:53:20.782963 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:53:20.782680    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k5kpb\" (UniqueName: \"kubernetes.io/projected/d4b8a3d9-c010-4a46-9f70-d9e5208a75e7-kube-api-access-k5kpb\") pod \"maas-api-e2e-mt-75c546e0-85b97d64d-8c64l\" (UID: \"d4b8a3d9-c010-4a46-9f70-d9e5208a75e7\") " pod="opendatahub/maas-api-e2e-mt-75c546e0-85b97d64d-8c64l"
Jul 06 22:53:20.884273 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:53:20.884160    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d4b8a3d9-c010-4a46-9f70-d9e5208a75e7-openshift-service-ca\") pod \"maas-api-e2e-mt-75c546e0-85b97d64d-8c64l\" (UID: \"d4b8a3d9-c010-4a46-9f70-d9e5208a75e7\") " pod="opendatahub/maas-api-e2e-mt-75c546e0-85b97d64d-8c64l"
Jul 06 22:53:20.884273 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:53:20.884229    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d4b8a3d9-c010-4a46-9f70-d9e5208a75e7-maas-api-tls\") pod \"maas-api-e2e-mt-75c546e0-85b97d64d-8c64l\" (UID: \"d4b8a3d9-c010-4a46-9f70-d9e5208a75e7\") " pod="opendatahub/maas-api-e2e-mt-75c546e0-85b97d64d-8c64l"
Jul 06 22:53:20.884456 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:53:20.884327    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k5kpb\" (UniqueName: \"kubernetes.io/projected/d4b8a3d9-c010-4a46-9f70-d9e5208a75e7-kube-api-access-k5kpb\") pod \"maas-api-e2e-mt-75c546e0-85b97d64d-8c64l\" (UID: \"d4b8a3d9-c010-4a46-9f70-d9e5208a75e7\") " pod="opendatahub/maas-api-e2e-mt-75c546e0-85b97d64d-8c64l"
Jul 06 22:53:20.884787 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:53:20.884758    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d4b8a3d9-c010-4a46-9f70-d9e5208a75e7-openshift-service-ca\") pod \"maas-api-e2e-mt-75c546e0-85b97d64d-8c64l\" (UID: \"d4b8a3d9-c010-4a46-9f70-d9e5208a75e7\") " pod="opendatahub/maas-api-e2e-mt-75c546e0-85b97d64d-8c64l"
Jul 06 22:53:20.886785 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:53:20.886759    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d4b8a3d9-c010-4a46-9f70-d9e5208a75e7-maas-api-tls\") pod \"maas-api-e2e-mt-75c546e0-85b97d64d-8c64l\" (UID: \"d4b8a3d9-c010-4a46-9f70-d9e5208a75e7\") " pod="opendatahub/maas-api-e2e-mt-75c546e0-85b97d64d-8c64l"
Jul 06 22:53:20.892422 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:53:20.892402    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k5kpb\" (UniqueName: \"kubernetes.io/projected/d4b8a3d9-c010-4a46-9f70-d9e5208a75e7-kube-api-access-k5kpb\") pod \"maas-api-e2e-mt-75c546e0-85b97d64d-8c64l\" (UID: \"d4b8a3d9-c010-4a46-9f70-d9e5208a75e7\") " pod="opendatahub/maas-api-e2e-mt-75c546e0-85b97d64d-8c64l"
Jul 06 22:53:20.935535 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:53:20.935501    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-75c546e0-85b97d64d-8c64l"
Jul 06 22:53:21.073651 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:53:21.073615    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-75c546e0-85b97d64d-8c64l"]
Jul 06 22:53:21.077312 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:53:21.077279    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4b8a3d9_c010_4a46_9f70_d9e5208a75e7.slice/crio-3b648a5da36986e46e002fb38a019d6bb7bbb0f2af5f68eb53a2b865229aa0b8 WatchSource:0}: Error finding container 3b648a5da36986e46e002fb38a019d6bb7bbb0f2af5f68eb53a2b865229aa0b8: Status 404 returned error can't find the container with id 3b648a5da36986e46e002fb38a019d6bb7bbb0f2af5f68eb53a2b865229aa0b8
Jul 06 22:53:22.070099 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:53:22.070066    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-75c546e0-85b97d64d-8c64l" event={"ID":"d4b8a3d9-c010-4a46-9f70-d9e5208a75e7","Type":"ContainerStarted","Data":"a6697bfc7880ec957fff4b3508371a15b70e024082cdfa2453a4db8259d3a22f"}
Jul 06 22:53:22.070504 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:53:22.070118    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-75c546e0-85b97d64d-8c64l" event={"ID":"d4b8a3d9-c010-4a46-9f70-d9e5208a75e7","Type":"ContainerStarted","Data":"3b648a5da36986e46e002fb38a019d6bb7bbb0f2af5f68eb53a2b865229aa0b8"}
Jul 06 22:53:22.070504 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:53:22.070181    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-mt-75c546e0-85b97d64d-8c64l"
Jul 06 22:53:22.088288 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:53:22.088234    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-mt-75c546e0-85b97d64d-8c64l" podStartSLOduration=1.734651167 podStartE2EDuration="2.088218625s" totalImagesPullingTime="353.567458ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:53:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:53:22.085680363 +0000 UTC m=+1767.510726906" watchObservedRunningTime="2026-07-06 22:53:22.088218625 +0000 UTC m=+1767.513265166"
Jul 06 22:53:23.269290 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:53:23.269249    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9q89m"]
Jul 06 22:53:28.079577 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:53:28.079549    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-mt-75c546e0-85b97d64d-8c64l"
Jul 06 22:53:37.578003 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:53:37.577967    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9q89m"]
Jul 06 22:53:55.131098 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:53:55.131071    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-t6jpk_03309f33-ff47-447d-956f-c89bdd4f8054/ovn-acl-logging/0.log"
Jul 06 22:53:55.131869 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:53:55.131850    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-t6jpk_03309f33-ff47-447d-956f-c89bdd4f8054/ovn-acl-logging/0.log"
Jul 06 22:54:05.367853 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:54:05.367814    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9q89m"]
Jul 06 22:54:08.183574 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:54:08.183526    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-75c546e0-85b97d64d-8c64l"]
Jul 06 22:54:08.184048 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:54:08.183775    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-mt-75c546e0-85b97d64d-8c64l" podUID="d4b8a3d9-c010-4a46-9f70-d9e5208a75e7" containerName="maas-api" containerID="cri-o://a6697bfc7880ec957fff4b3508371a15b70e024082cdfa2453a4db8259d3a22f" gracePeriod=30
Jul 06 22:54:08.437772 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:54:08.437717    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-75c546e0-85b97d64d-8c64l"
Jul 06 22:54:08.493845 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:54:08.493822    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d4b8a3d9-c010-4a46-9f70-d9e5208a75e7-maas-api-tls\") pod \"d4b8a3d9-c010-4a46-9f70-d9e5208a75e7\" (UID: \"d4b8a3d9-c010-4a46-9f70-d9e5208a75e7\") "
Jul 06 22:54:08.494036 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:54:08.493875    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k5kpb\" (UniqueName: \"kubernetes.io/projected/d4b8a3d9-c010-4a46-9f70-d9e5208a75e7-kube-api-access-k5kpb\") pod \"d4b8a3d9-c010-4a46-9f70-d9e5208a75e7\" (UID: \"d4b8a3d9-c010-4a46-9f70-d9e5208a75e7\") "
Jul 06 22:54:08.494036 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:54:08.493957    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d4b8a3d9-c010-4a46-9f70-d9e5208a75e7-openshift-service-ca\") pod \"d4b8a3d9-c010-4a46-9f70-d9e5208a75e7\" (UID: \"d4b8a3d9-c010-4a46-9f70-d9e5208a75e7\") "
Jul 06 22:54:08.494374 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:54:08.494349    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d4b8a3d9-c010-4a46-9f70-d9e5208a75e7-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d4b8a3d9-c010-4a46-9f70-d9e5208a75e7" (UID: "d4b8a3d9-c010-4a46-9f70-d9e5208a75e7"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 22:54:08.496102 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:54:08.496077    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d4b8a3d9-c010-4a46-9f70-d9e5208a75e7-kube-api-access-k5kpb" (OuterVolumeSpecName: "kube-api-access-k5kpb") pod "d4b8a3d9-c010-4a46-9f70-d9e5208a75e7" (UID: "d4b8a3d9-c010-4a46-9f70-d9e5208a75e7"). InnerVolumeSpecName "kube-api-access-k5kpb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 22:54:08.496257 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:54:08.496193    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d4b8a3d9-c010-4a46-9f70-d9e5208a75e7-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d4b8a3d9-c010-4a46-9f70-d9e5208a75e7" (UID: "d4b8a3d9-c010-4a46-9f70-d9e5208a75e7"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 22:54:08.595243 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:54:08.595204    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d4b8a3d9-c010-4a46-9f70-d9e5208a75e7-openshift-service-ca\") on node \"ip-10-0-133-105.ec2.internal\" DevicePath \"\""
Jul 06 22:54:08.595243 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:54:08.595241    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d4b8a3d9-c010-4a46-9f70-d9e5208a75e7-maas-api-tls\") on node \"ip-10-0-133-105.ec2.internal\" DevicePath \"\""
Jul 06 22:54:08.595243 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:54:08.595251    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k5kpb\" (UniqueName: \"kubernetes.io/projected/d4b8a3d9-c010-4a46-9f70-d9e5208a75e7-kube-api-access-k5kpb\") on node \"ip-10-0-133-105.ec2.internal\" DevicePath \"\""
Jul 06 22:54:09.296538 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:54:09.296502    2563 generic.go:356] "Generic (PLEG): container finished" podID="d4b8a3d9-c010-4a46-9f70-d9e5208a75e7" containerID="a6697bfc7880ec957fff4b3508371a15b70e024082cdfa2453a4db8259d3a22f" exitCode=0
Jul 06 22:54:09.296538 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:54:09.296540    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-75c546e0-85b97d64d-8c64l" event={"ID":"d4b8a3d9-c010-4a46-9f70-d9e5208a75e7","Type":"ContainerDied","Data":"a6697bfc7880ec957fff4b3508371a15b70e024082cdfa2453a4db8259d3a22f"}
Jul 06 22:54:09.297059 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:54:09.296550    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-75c546e0-85b97d64d-8c64l"
Jul 06 22:54:09.297059 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:54:09.296580    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-75c546e0-85b97d64d-8c64l" event={"ID":"d4b8a3d9-c010-4a46-9f70-d9e5208a75e7","Type":"ContainerDied","Data":"3b648a5da36986e46e002fb38a019d6bb7bbb0f2af5f68eb53a2b865229aa0b8"}
Jul 06 22:54:09.297059 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:54:09.296603    2563 scope.go:117] "RemoveContainer" containerID="a6697bfc7880ec957fff4b3508371a15b70e024082cdfa2453a4db8259d3a22f"
Jul 06 22:54:09.305488 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:54:09.305470    2563 scope.go:117] "RemoveContainer" containerID="a6697bfc7880ec957fff4b3508371a15b70e024082cdfa2453a4db8259d3a22f"
Jul 06 22:54:09.305720 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:54:09.305701    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a6697bfc7880ec957fff4b3508371a15b70e024082cdfa2453a4db8259d3a22f\": container with ID starting with a6697bfc7880ec957fff4b3508371a15b70e024082cdfa2453a4db8259d3a22f not found: ID does not exist" containerID="a6697bfc7880ec957fff4b3508371a15b70e024082cdfa2453a4db8259d3a22f"
Jul 06 22:54:09.305769 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:54:09.305727    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a6697bfc7880ec957fff4b3508371a15b70e024082cdfa2453a4db8259d3a22f"} err="failed to get container status \"a6697bfc7880ec957fff4b3508371a15b70e024082cdfa2453a4db8259d3a22f\": rpc error: code = NotFound desc = could not find container \"a6697bfc7880ec957fff4b3508371a15b70e024082cdfa2453a4db8259d3a22f\": container with ID starting with a6697bfc7880ec957fff4b3508371a15b70e024082cdfa2453a4db8259d3a22f not found: ID does not exist"
Jul 06 22:54:09.312923 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:54:09.312900    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-75c546e0-85b97d64d-8c64l"]
Jul 06 22:54:09.317053 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:54:09.317032    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-mt-75c546e0-85b97d64d-8c64l"]
Jul 06 22:54:11.079922 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:54:11.079886    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d4b8a3d9-c010-4a46-9f70-d9e5208a75e7" path="/var/lib/kubelet/pods/d4b8a3d9-c010-4a46-9f70-d9e5208a75e7/volumes"
Jul 06 22:54:18.404717 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:54:18.404679    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9q89m"]
Jul 06 22:54:59.267482 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:54:59.267440    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9q89m"]
Jul 06 22:56:01.314829 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:56:01.314778    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-inf-a-8f9c60-64ccc46bbc-c57r7"]
Jul 06 22:56:01.315375 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:56:01.315358    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d4b8a3d9-c010-4a46-9f70-d9e5208a75e7" containerName="maas-api"
Jul 06 22:56:01.315437 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:56:01.315379    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="d4b8a3d9-c010-4a46-9f70-d9e5208a75e7" containerName="maas-api"
Jul 06 22:56:01.315504 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:56:01.315492    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="d4b8a3d9-c010-4a46-9f70-d9e5208a75e7" containerName="maas-api"
Jul 06 22:56:01.318749 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:56:01.318725    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-8f9c60-64ccc46bbc-c57r7"
Jul 06 22:56:01.321365 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:56:01.321339    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-f7f72\""
Jul 06 22:56:01.321488 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:56:01.321390    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-inf-a-8f9c60-serving-cert\""
Jul 06 22:56:01.331213 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:56:01.331189    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-8f9c60-64ccc46bbc-c57r7"]
Jul 06 22:56:01.472254 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:56:01.472221    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h2qvf\" (UniqueName: \"kubernetes.io/projected/1681f8bb-f731-41d7-86b9-dbc399f4f930-kube-api-access-h2qvf\") pod \"maas-api-e2e-inf-a-8f9c60-64ccc46bbc-c57r7\" (UID: \"1681f8bb-f731-41d7-86b9-dbc399f4f930\") " pod="opendatahub/maas-api-e2e-inf-a-8f9c60-64ccc46bbc-c57r7"
Jul 06 22:56:01.472449 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:56:01.472275    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1681f8bb-f731-41d7-86b9-dbc399f4f930-maas-api-tls\") pod \"maas-api-e2e-inf-a-8f9c60-64ccc46bbc-c57r7\" (UID: \"1681f8bb-f731-41d7-86b9-dbc399f4f930\") " pod="opendatahub/maas-api-e2e-inf-a-8f9c60-64ccc46bbc-c57r7"
Jul 06 22:56:01.472449 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:56:01.472300    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1681f8bb-f731-41d7-86b9-dbc399f4f930-openshift-service-ca\") pod \"maas-api-e2e-inf-a-8f9c60-64ccc46bbc-c57r7\" (UID: \"1681f8bb-f731-41d7-86b9-dbc399f4f930\") " pod="opendatahub/maas-api-e2e-inf-a-8f9c60-64ccc46bbc-c57r7"
Jul 06 22:56:01.573817 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:56:01.573733    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h2qvf\" (UniqueName: \"kubernetes.io/projected/1681f8bb-f731-41d7-86b9-dbc399f4f930-kube-api-access-h2qvf\") pod \"maas-api-e2e-inf-a-8f9c60-64ccc46bbc-c57r7\" (UID: \"1681f8bb-f731-41d7-86b9-dbc399f4f930\") " pod="opendatahub/maas-api-e2e-inf-a-8f9c60-64ccc46bbc-c57r7"
Jul 06 22:56:01.573817 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:56:01.573786    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1681f8bb-f731-41d7-86b9-dbc399f4f930-maas-api-tls\") pod \"maas-api-e2e-inf-a-8f9c60-64ccc46bbc-c57r7\" (UID: \"1681f8bb-f731-41d7-86b9-dbc399f4f930\") " pod="opendatahub/maas-api-e2e-inf-a-8f9c60-64ccc46bbc-c57r7"
Jul 06 22:56:01.574025 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:56:01.573822    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1681f8bb-f731-41d7-86b9-dbc399f4f930-openshift-service-ca\") pod \"maas-api-e2e-inf-a-8f9c60-64ccc46bbc-c57r7\" (UID: \"1681f8bb-f731-41d7-86b9-dbc399f4f930\") " pod="opendatahub/maas-api-e2e-inf-a-8f9c60-64ccc46bbc-c57r7"
Jul 06 22:56:01.574113 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:56:01.574093    2563 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-inf-a-8f9c60-serving-cert: secret "maas-api-e2e-inf-a-8f9c60-serving-cert" not found
Jul 06 22:56:01.574210 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:56:01.574200    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1681f8bb-f731-41d7-86b9-dbc399f4f930-maas-api-tls podName:1681f8bb-f731-41d7-86b9-dbc399f4f930 nodeName:}" failed. No retries permitted until 2026-07-06 22:56:02.074149947 +0000 UTC m=+1927.499196473 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/1681f8bb-f731-41d7-86b9-dbc399f4f930-maas-api-tls") pod "maas-api-e2e-inf-a-8f9c60-64ccc46bbc-c57r7" (UID: "1681f8bb-f731-41d7-86b9-dbc399f4f930") : secret "maas-api-e2e-inf-a-8f9c60-serving-cert" not found
Jul 06 22:56:01.574456 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:56:01.574434    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1681f8bb-f731-41d7-86b9-dbc399f4f930-openshift-service-ca\") pod \"maas-api-e2e-inf-a-8f9c60-64ccc46bbc-c57r7\" (UID: \"1681f8bb-f731-41d7-86b9-dbc399f4f930\") " pod="opendatahub/maas-api-e2e-inf-a-8f9c60-64ccc46bbc-c57r7"
Jul 06 22:56:01.584852 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:56:01.584832    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h2qvf\" (UniqueName: \"kubernetes.io/projected/1681f8bb-f731-41d7-86b9-dbc399f4f930-kube-api-access-h2qvf\") pod \"maas-api-e2e-inf-a-8f9c60-64ccc46bbc-c57r7\" (UID: \"1681f8bb-f731-41d7-86b9-dbc399f4f930\") " pod="opendatahub/maas-api-e2e-inf-a-8f9c60-64ccc46bbc-c57r7"
Jul 06 22:56:02.077967 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:56:02.077929    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1681f8bb-f731-41d7-86b9-dbc399f4f930-maas-api-tls\") pod \"maas-api-e2e-inf-a-8f9c60-64ccc46bbc-c57r7\" (UID: \"1681f8bb-f731-41d7-86b9-dbc399f4f930\") " pod="opendatahub/maas-api-e2e-inf-a-8f9c60-64ccc46bbc-c57r7"
Jul 06 22:56:02.080640 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:56:02.080618    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1681f8bb-f731-41d7-86b9-dbc399f4f930-maas-api-tls\") pod \"maas-api-e2e-inf-a-8f9c60-64ccc46bbc-c57r7\" (UID: \"1681f8bb-f731-41d7-86b9-dbc399f4f930\") " pod="opendatahub/maas-api-e2e-inf-a-8f9c60-64ccc46bbc-c57r7"
Jul 06 22:56:02.230356 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:56:02.230311    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-8f9c60-64ccc46bbc-c57r7"
Jul 06 22:56:02.370464 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:56:02.370439    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-8f9c60-64ccc46bbc-c57r7"]
Jul 06 22:56:02.373152 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:56:02.373124    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1681f8bb_f731_41d7_86b9_dbc399f4f930.slice/crio-8b16f6c262fc2e45ced1ea9e33ecad5c13dc26b2a0584aa49a7fcd970d780ae2 WatchSource:0}: Error finding container 8b16f6c262fc2e45ced1ea9e33ecad5c13dc26b2a0584aa49a7fcd970d780ae2: Status 404 returned error can't find the container with id 8b16f6c262fc2e45ced1ea9e33ecad5c13dc26b2a0584aa49a7fcd970d780ae2
Jul 06 22:56:02.374672 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:56:02.374653    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 06 22:56:02.837043 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:56:02.837010    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-8f9c60-64ccc46bbc-c57r7" event={"ID":"1681f8bb-f731-41d7-86b9-dbc399f4f930","Type":"ContainerStarted","Data":"8b16f6c262fc2e45ced1ea9e33ecad5c13dc26b2a0584aa49a7fcd970d780ae2"}
Jul 06 22:56:03.845255 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:56:03.845219    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-8f9c60-64ccc46bbc-c57r7" event={"ID":"1681f8bb-f731-41d7-86b9-dbc399f4f930","Type":"ContainerStarted","Data":"773af0abeb0482471490fa4a8b68a9c0e49c7c00ca83eb95d541949c84295408"}
Jul 06 22:56:03.845660 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:56:03.845294    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-inf-a-8f9c60-64ccc46bbc-c57r7"
Jul 06 22:56:03.860353 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:56:03.860299    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-inf-a-8f9c60-64ccc46bbc-c57r7" podStartSLOduration=2.379759993 podStartE2EDuration="2.860279944s" totalImagesPullingTime="480.519951ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:56:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:56:03.859928857 +0000 UTC m=+1929.284975399" watchObservedRunningTime="2026-07-06 22:56:03.860279944 +0000 UTC m=+1929.285326486"
Jul 06 22:56:08.568372 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:56:08.568330    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9q89m"]
Jul 06 22:56:09.394256 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:56:09.394225    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-inf-b-878700-9bc76d954-jkxnj"]
Jul 06 22:56:09.397956 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:56:09.397939    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-878700-9bc76d954-jkxnj"
Jul 06 22:56:09.400617 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:56:09.400596    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-inf-b-878700-serving-cert\""
Jul 06 22:56:09.407432 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:56:09.407407    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-878700-9bc76d954-jkxnj"]
Jul 06 22:56:09.547437 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:56:09.547407    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/94a7e22c-1842-43b5-aaf9-006011bcb49c-openshift-service-ca\") pod \"maas-api-e2e-inf-b-878700-9bc76d954-jkxnj\" (UID: \"94a7e22c-1842-43b5-aaf9-006011bcb49c\") " pod="opendatahub/maas-api-e2e-inf-b-878700-9bc76d954-jkxnj"
Jul 06 22:56:09.547605 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:56:09.547475    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/94a7e22c-1842-43b5-aaf9-006011bcb49c-maas-api-tls\") pod \"maas-api-e2e-inf-b-878700-9bc76d954-jkxnj\" (UID: \"94a7e22c-1842-43b5-aaf9-006011bcb49c\") " pod="opendatahub/maas-api-e2e-inf-b-878700-9bc76d954-jkxnj"
Jul 06 22:56:09.547605 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:56:09.547526    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x7nrx\" (UniqueName: \"kubernetes.io/projected/94a7e22c-1842-43b5-aaf9-006011bcb49c-kube-api-access-x7nrx\") pod \"maas-api-e2e-inf-b-878700-9bc76d954-jkxnj\" (UID: \"94a7e22c-1842-43b5-aaf9-006011bcb49c\") " pod="opendatahub/maas-api-e2e-inf-b-878700-9bc76d954-jkxnj"
Jul 06 22:56:09.648190 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:56:09.648083    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/94a7e22c-1842-43b5-aaf9-006011bcb49c-openshift-service-ca\") pod \"maas-api-e2e-inf-b-878700-9bc76d954-jkxnj\" (UID: \"94a7e22c-1842-43b5-aaf9-006011bcb49c\") " pod="opendatahub/maas-api-e2e-inf-b-878700-9bc76d954-jkxnj"
Jul 06 22:56:09.648190 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:56:09.648157    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/94a7e22c-1842-43b5-aaf9-006011bcb49c-maas-api-tls\") pod \"maas-api-e2e-inf-b-878700-9bc76d954-jkxnj\" (UID: \"94a7e22c-1842-43b5-aaf9-006011bcb49c\") " pod="opendatahub/maas-api-e2e-inf-b-878700-9bc76d954-jkxnj"
Jul 06 22:56:09.648614 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:56:09.648233    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x7nrx\" (UniqueName: \"kubernetes.io/projected/94a7e22c-1842-43b5-aaf9-006011bcb49c-kube-api-access-x7nrx\") pod \"maas-api-e2e-inf-b-878700-9bc76d954-jkxnj\" (UID: \"94a7e22c-1842-43b5-aaf9-006011bcb49c\") " pod="opendatahub/maas-api-e2e-inf-b-878700-9bc76d954-jkxnj"
Jul 06 22:56:09.648790 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:56:09.648769    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/94a7e22c-1842-43b5-aaf9-006011bcb49c-openshift-service-ca\") pod \"maas-api-e2e-inf-b-878700-9bc76d954-jkxnj\" (UID: \"94a7e22c-1842-43b5-aaf9-006011bcb49c\") " pod="opendatahub/maas-api-e2e-inf-b-878700-9bc76d954-jkxnj"
Jul 06 22:56:09.650863 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:56:09.650833    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/94a7e22c-1842-43b5-aaf9-006011bcb49c-maas-api-tls\") pod \"maas-api-e2e-inf-b-878700-9bc76d954-jkxnj\" (UID: \"94a7e22c-1842-43b5-aaf9-006011bcb49c\") " pod="opendatahub/maas-api-e2e-inf-b-878700-9bc76d954-jkxnj"
Jul 06 22:56:09.659376 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:56:09.659350    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x7nrx\" (UniqueName: \"kubernetes.io/projected/94a7e22c-1842-43b5-aaf9-006011bcb49c-kube-api-access-x7nrx\") pod \"maas-api-e2e-inf-b-878700-9bc76d954-jkxnj\" (UID: \"94a7e22c-1842-43b5-aaf9-006011bcb49c\") " pod="opendatahub/maas-api-e2e-inf-b-878700-9bc76d954-jkxnj"
Jul 06 22:56:09.711337 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:56:09.711303    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-878700-9bc76d954-jkxnj"
Jul 06 22:56:09.855358 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:56:09.855331    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-inf-a-8f9c60-64ccc46bbc-c57r7"
Jul 06 22:56:09.861413 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:56:09.861365    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-878700-9bc76d954-jkxnj"]
Jul 06 22:56:09.864338 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:56:09.864311    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod94a7e22c_1842_43b5_aaf9_006011bcb49c.slice/crio-f3285700c8270598ce4b9729fc2e7282657c25ac04ddf153a7c0899b584cd0d7 WatchSource:0}: Error finding container f3285700c8270598ce4b9729fc2e7282657c25ac04ddf153a7c0899b584cd0d7: Status 404 returned error can't find the container with id f3285700c8270598ce4b9729fc2e7282657c25ac04ddf153a7c0899b584cd0d7
Jul 06 22:56:09.873608 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:56:09.873582    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-878700-9bc76d954-jkxnj" event={"ID":"94a7e22c-1842-43b5-aaf9-006011bcb49c","Type":"ContainerStarted","Data":"f3285700c8270598ce4b9729fc2e7282657c25ac04ddf153a7c0899b584cd0d7"}
Jul 06 22:56:10.885895 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:56:10.885854    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-878700-9bc76d954-jkxnj" event={"ID":"94a7e22c-1842-43b5-aaf9-006011bcb49c","Type":"ContainerStarted","Data":"a393975cf8331d383197fcdf70fdc63109a12e92fc88527ebb4fab089447152b"}
Jul 06 22:56:10.886400 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:56:10.885996    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-inf-b-878700-9bc76d954-jkxnj"
Jul 06 22:56:10.904209 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:56:10.904143    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-inf-b-878700-9bc76d954-jkxnj" podStartSLOduration=1.548836959 podStartE2EDuration="1.904128984s" totalImagesPullingTime="355.292025ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:56:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:56:10.903084867 +0000 UTC m=+1936.328131409" watchObservedRunningTime="2026-07-06 22:56:10.904128984 +0000 UTC m=+1936.329175525"
Jul 06 22:56:16.896187 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:56:16.896151    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-inf-b-878700-9bc76d954-jkxnj"
Jul 06 22:56:17.974964 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:56:17.974921    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9q89m"]
Jul 06 22:56:30.880926 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:56:30.880889    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9q89m"]
Jul 06 22:57:01.888601 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:57:01.888559    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9q89m"]
Jul 06 22:57:16.572685 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:57:16.572652    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9q89m"]
Jul 06 22:57:34.598442 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:57:34.598410    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-8f9c60-64ccc46bbc-c57r7"]
Jul 06 22:57:34.599003 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:57:34.598658    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-inf-a-8f9c60-64ccc46bbc-c57r7" podUID="1681f8bb-f731-41d7-86b9-dbc399f4f930" containerName="maas-api" containerID="cri-o://773af0abeb0482471490fa4a8b68a9c0e49c7c00ca83eb95d541949c84295408" gracePeriod=30
Jul 06 22:57:34.853948 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:57:34.853891    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-8f9c60-64ccc46bbc-c57r7"
Jul 06 22:57:35.004092 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:57:35.004065    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1681f8bb-f731-41d7-86b9-dbc399f4f930-maas-api-tls\") pod \"1681f8bb-f731-41d7-86b9-dbc399f4f930\" (UID: \"1681f8bb-f731-41d7-86b9-dbc399f4f930\") "
Jul 06 22:57:35.004262 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:57:35.004191    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1681f8bb-f731-41d7-86b9-dbc399f4f930-openshift-service-ca\") pod \"1681f8bb-f731-41d7-86b9-dbc399f4f930\" (UID: \"1681f8bb-f731-41d7-86b9-dbc399f4f930\") "
Jul 06 22:57:35.004262 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:57:35.004212    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h2qvf\" (UniqueName: \"kubernetes.io/projected/1681f8bb-f731-41d7-86b9-dbc399f4f930-kube-api-access-h2qvf\") pod \"1681f8bb-f731-41d7-86b9-dbc399f4f930\" (UID: \"1681f8bb-f731-41d7-86b9-dbc399f4f930\") "
Jul 06 22:57:35.004528 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:57:35.004505    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1681f8bb-f731-41d7-86b9-dbc399f4f930-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "1681f8bb-f731-41d7-86b9-dbc399f4f930" (UID: "1681f8bb-f731-41d7-86b9-dbc399f4f930"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 22:57:35.006252 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:57:35.006221    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1681f8bb-f731-41d7-86b9-dbc399f4f930-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "1681f8bb-f731-41d7-86b9-dbc399f4f930" (UID: "1681f8bb-f731-41d7-86b9-dbc399f4f930"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 22:57:35.006365 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:57:35.006306    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1681f8bb-f731-41d7-86b9-dbc399f4f930-kube-api-access-h2qvf" (OuterVolumeSpecName: "kube-api-access-h2qvf") pod "1681f8bb-f731-41d7-86b9-dbc399f4f930" (UID: "1681f8bb-f731-41d7-86b9-dbc399f4f930"). InnerVolumeSpecName "kube-api-access-h2qvf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 22:57:35.105318 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:57:35.105254    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1681f8bb-f731-41d7-86b9-dbc399f4f930-openshift-service-ca\") on node \"ip-10-0-133-105.ec2.internal\" DevicePath \"\""
Jul 06 22:57:35.105318 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:57:35.105277    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h2qvf\" (UniqueName: \"kubernetes.io/projected/1681f8bb-f731-41d7-86b9-dbc399f4f930-kube-api-access-h2qvf\") on node \"ip-10-0-133-105.ec2.internal\" DevicePath \"\""
Jul 06 22:57:35.105318 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:57:35.105288    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1681f8bb-f731-41d7-86b9-dbc399f4f930-maas-api-tls\") on node \"ip-10-0-133-105.ec2.internal\" DevicePath \"\""
Jul 06 22:57:35.291466 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:57:35.291431    2563 generic.go:356] "Generic (PLEG): container finished" podID="1681f8bb-f731-41d7-86b9-dbc399f4f930" containerID="773af0abeb0482471490fa4a8b68a9c0e49c7c00ca83eb95d541949c84295408" exitCode=0
Jul 06 22:57:35.291666 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:57:35.291479    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-8f9c60-64ccc46bbc-c57r7"
Jul 06 22:57:35.291666 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:57:35.291501    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-8f9c60-64ccc46bbc-c57r7" event={"ID":"1681f8bb-f731-41d7-86b9-dbc399f4f930","Type":"ContainerDied","Data":"773af0abeb0482471490fa4a8b68a9c0e49c7c00ca83eb95d541949c84295408"}
Jul 06 22:57:35.291666 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:57:35.291555    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-8f9c60-64ccc46bbc-c57r7" event={"ID":"1681f8bb-f731-41d7-86b9-dbc399f4f930","Type":"ContainerDied","Data":"8b16f6c262fc2e45ced1ea9e33ecad5c13dc26b2a0584aa49a7fcd970d780ae2"}
Jul 06 22:57:35.291666 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:57:35.291585    2563 scope.go:117] "RemoveContainer" containerID="773af0abeb0482471490fa4a8b68a9c0e49c7c00ca83eb95d541949c84295408"
Jul 06 22:57:35.300693 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:57:35.300673    2563 scope.go:117] "RemoveContainer" containerID="773af0abeb0482471490fa4a8b68a9c0e49c7c00ca83eb95d541949c84295408"
Jul 06 22:57:35.300932 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:57:35.300914    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"773af0abeb0482471490fa4a8b68a9c0e49c7c00ca83eb95d541949c84295408\": container with ID starting with 773af0abeb0482471490fa4a8b68a9c0e49c7c00ca83eb95d541949c84295408 not found: ID does not exist" containerID="773af0abeb0482471490fa4a8b68a9c0e49c7c00ca83eb95d541949c84295408"
Jul 06 22:57:35.300975 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:57:35.300941    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"773af0abeb0482471490fa4a8b68a9c0e49c7c00ca83eb95d541949c84295408"} err="failed to get container status \"773af0abeb0482471490fa4a8b68a9c0e49c7c00ca83eb95d541949c84295408\": rpc error: code = NotFound desc = could not find container \"773af0abeb0482471490fa4a8b68a9c0e49c7c00ca83eb95d541949c84295408\": container with ID starting with 773af0abeb0482471490fa4a8b68a9c0e49c7c00ca83eb95d541949c84295408 not found: ID does not exist"
Jul 06 22:57:35.307656 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:57:35.307632    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-8f9c60-64ccc46bbc-c57r7"]
Jul 06 22:57:35.309476 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:57:35.309454    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-8f9c60-64ccc46bbc-c57r7"]
Jul 06 22:57:37.079687 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:57:37.079652    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1681f8bb-f731-41d7-86b9-dbc399f4f930" path="/var/lib/kubelet/pods/1681f8bb-f731-41d7-86b9-dbc399f4f930/volumes"
Jul 06 22:57:37.570851 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:57:37.570814    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9q89m"]
Jul 06 22:57:37.851099 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:57:37.851011    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="opendatahub/maas-api-e2e-inf-a-8f9c60-64ccc46bbc-c57r7" podUID="1681f8bb-f731-41d7-86b9-dbc399f4f930" containerName="maas-api" probeResult="failure" output="Get \"https://10.133.0.40:8443/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Jul 06 22:58:17.864003 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:58:17.863918    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-878700-9bc76d954-jkxnj"]
Jul 06 22:58:17.864575 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:58:17.864113    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-inf-b-878700-9bc76d954-jkxnj" podUID="94a7e22c-1842-43b5-aaf9-006011bcb49c" containerName="maas-api" containerID="cri-o://a393975cf8331d383197fcdf70fdc63109a12e92fc88527ebb4fab089447152b" gracePeriod=30
Jul 06 22:58:18.122371 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:58:18.122302    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-878700-9bc76d954-jkxnj"
Jul 06 22:58:18.187952 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:58:18.187919    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/94a7e22c-1842-43b5-aaf9-006011bcb49c-openshift-service-ca\") pod \"94a7e22c-1842-43b5-aaf9-006011bcb49c\" (UID: \"94a7e22c-1842-43b5-aaf9-006011bcb49c\") "
Jul 06 22:58:18.188134 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:58:18.187980    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x7nrx\" (UniqueName: \"kubernetes.io/projected/94a7e22c-1842-43b5-aaf9-006011bcb49c-kube-api-access-x7nrx\") pod \"94a7e22c-1842-43b5-aaf9-006011bcb49c\" (UID: \"94a7e22c-1842-43b5-aaf9-006011bcb49c\") "
Jul 06 22:58:18.188134 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:58:18.188051    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/94a7e22c-1842-43b5-aaf9-006011bcb49c-maas-api-tls\") pod \"94a7e22c-1842-43b5-aaf9-006011bcb49c\" (UID: \"94a7e22c-1842-43b5-aaf9-006011bcb49c\") "
Jul 06 22:58:18.188354 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:58:18.188326    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/94a7e22c-1842-43b5-aaf9-006011bcb49c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "94a7e22c-1842-43b5-aaf9-006011bcb49c" (UID: "94a7e22c-1842-43b5-aaf9-006011bcb49c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 22:58:18.190291 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:58:18.190267    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/94a7e22c-1842-43b5-aaf9-006011bcb49c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "94a7e22c-1842-43b5-aaf9-006011bcb49c" (UID: "94a7e22c-1842-43b5-aaf9-006011bcb49c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 22:58:18.190291 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:58:18.190278    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/94a7e22c-1842-43b5-aaf9-006011bcb49c-kube-api-access-x7nrx" (OuterVolumeSpecName: "kube-api-access-x7nrx") pod "94a7e22c-1842-43b5-aaf9-006011bcb49c" (UID: "94a7e22c-1842-43b5-aaf9-006011bcb49c"). InnerVolumeSpecName "kube-api-access-x7nrx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 22:58:18.288929 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:58:18.288892    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/94a7e22c-1842-43b5-aaf9-006011bcb49c-openshift-service-ca\") on node \"ip-10-0-133-105.ec2.internal\" DevicePath \"\""
Jul 06 22:58:18.288929 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:58:18.288920    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x7nrx\" (UniqueName: \"kubernetes.io/projected/94a7e22c-1842-43b5-aaf9-006011bcb49c-kube-api-access-x7nrx\") on node \"ip-10-0-133-105.ec2.internal\" DevicePath \"\""
Jul 06 22:58:18.288929 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:58:18.288936    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/94a7e22c-1842-43b5-aaf9-006011bcb49c-maas-api-tls\") on node \"ip-10-0-133-105.ec2.internal\" DevicePath \"\""
Jul 06 22:58:18.496072 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:58:18.496040    2563 generic.go:356] "Generic (PLEG): container finished" podID="94a7e22c-1842-43b5-aaf9-006011bcb49c" containerID="a393975cf8331d383197fcdf70fdc63109a12e92fc88527ebb4fab089447152b" exitCode=0
Jul 06 22:58:18.496276 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:58:18.496085    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-878700-9bc76d954-jkxnj"
Jul 06 22:58:18.496276 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:58:18.496138    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-878700-9bc76d954-jkxnj" event={"ID":"94a7e22c-1842-43b5-aaf9-006011bcb49c","Type":"ContainerDied","Data":"a393975cf8331d383197fcdf70fdc63109a12e92fc88527ebb4fab089447152b"}
Jul 06 22:58:18.496276 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:58:18.496216    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-878700-9bc76d954-jkxnj" event={"ID":"94a7e22c-1842-43b5-aaf9-006011bcb49c","Type":"ContainerDied","Data":"f3285700c8270598ce4b9729fc2e7282657c25ac04ddf153a7c0899b584cd0d7"}
Jul 06 22:58:18.496276 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:58:18.496248    2563 scope.go:117] "RemoveContainer" containerID="a393975cf8331d383197fcdf70fdc63109a12e92fc88527ebb4fab089447152b"
Jul 06 22:58:18.505110 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:58:18.505095    2563 scope.go:117] "RemoveContainer" containerID="a393975cf8331d383197fcdf70fdc63109a12e92fc88527ebb4fab089447152b"
Jul 06 22:58:18.505373 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:58:18.505352    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a393975cf8331d383197fcdf70fdc63109a12e92fc88527ebb4fab089447152b\": container with ID starting with a393975cf8331d383197fcdf70fdc63109a12e92fc88527ebb4fab089447152b not found: ID does not exist" containerID="a393975cf8331d383197fcdf70fdc63109a12e92fc88527ebb4fab089447152b"
Jul 06 22:58:18.505465 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:58:18.505378    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a393975cf8331d383197fcdf70fdc63109a12e92fc88527ebb4fab089447152b"} err="failed to get container status \"a393975cf8331d383197fcdf70fdc63109a12e92fc88527ebb4fab089447152b\": rpc error: code = NotFound desc = could not find container \"a393975cf8331d383197fcdf70fdc63109a12e92fc88527ebb4fab089447152b\": container with ID starting with a393975cf8331d383197fcdf70fdc63109a12e92fc88527ebb4fab089447152b not found: ID does not exist"
Jul 06 22:58:18.518396 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:58:18.518376    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-878700-9bc76d954-jkxnj"]
Jul 06 22:58:18.520347 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:58:18.520328    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-878700-9bc76d954-jkxnj"]
Jul 06 22:58:19.085403 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:58:19.085375    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="94a7e22c-1842-43b5-aaf9-006011bcb49c" path="/var/lib/kubelet/pods/94a7e22c-1842-43b5-aaf9-006011bcb49c/volumes"
Jul 06 22:58:20.870561 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:58:20.870527    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9q89m"]
Jul 06 22:58:28.049362 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:58:28.049331    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-api-a-8090dc-78bb959dd8-qzn96"]
Jul 06 22:58:28.049725 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:58:28.049712    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1681f8bb-f731-41d7-86b9-dbc399f4f930" containerName="maas-api"
Jul 06 22:58:28.049771 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:58:28.049726    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="1681f8bb-f731-41d7-86b9-dbc399f4f930" containerName="maas-api"
Jul 06 22:58:28.049771 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:58:28.049739    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="94a7e22c-1842-43b5-aaf9-006011bcb49c" containerName="maas-api"
Jul 06 22:58:28.049771 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:58:28.049744    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="94a7e22c-1842-43b5-aaf9-006011bcb49c" containerName="maas-api"
Jul 06 22:58:28.049870 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:58:28.049797    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="1681f8bb-f731-41d7-86b9-dbc399f4f930" containerName="maas-api"
Jul 06 22:58:28.049870 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:58:28.049805    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="94a7e22c-1842-43b5-aaf9-006011bcb49c" containerName="maas-api"
Jul 06 22:58:28.052796 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:58:28.052775    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-8090dc-78bb959dd8-qzn96"
Jul 06 22:58:28.056159 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:58:28.056120    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-api-a-8090dc-serving-cert\""
Jul 06 22:58:28.056310 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:58:28.056225    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-f7f72\""
Jul 06 22:58:28.065670 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:58:28.065645    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-a-8090dc-78bb959dd8-qzn96"]
Jul 06 22:58:28.179312 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:58:28.179280    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-766j2\" (UniqueName: \"kubernetes.io/projected/f601a5d4-b3fb-4109-b257-b390871c8197-kube-api-access-766j2\") pod \"maas-api-e2e-api-a-8090dc-78bb959dd8-qzn96\" (UID: \"f601a5d4-b3fb-4109-b257-b390871c8197\") " pod="opendatahub/maas-api-e2e-api-a-8090dc-78bb959dd8-qzn96"
Jul 06 22:58:28.179497 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:58:28.179321    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f601a5d4-b3fb-4109-b257-b390871c8197-openshift-service-ca\") pod \"maas-api-e2e-api-a-8090dc-78bb959dd8-qzn96\" (UID: \"f601a5d4-b3fb-4109-b257-b390871c8197\") " pod="opendatahub/maas-api-e2e-api-a-8090dc-78bb959dd8-qzn96"
Jul 06 22:58:28.179497 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:58:28.179345    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f601a5d4-b3fb-4109-b257-b390871c8197-maas-api-tls\") pod \"maas-api-e2e-api-a-8090dc-78bb959dd8-qzn96\" (UID: \"f601a5d4-b3fb-4109-b257-b390871c8197\") " pod="opendatahub/maas-api-e2e-api-a-8090dc-78bb959dd8-qzn96"
Jul 06 22:58:28.280791 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:58:28.280754    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-766j2\" (UniqueName: \"kubernetes.io/projected/f601a5d4-b3fb-4109-b257-b390871c8197-kube-api-access-766j2\") pod \"maas-api-e2e-api-a-8090dc-78bb959dd8-qzn96\" (UID: \"f601a5d4-b3fb-4109-b257-b390871c8197\") " pod="opendatahub/maas-api-e2e-api-a-8090dc-78bb959dd8-qzn96"
Jul 06 22:58:28.280970 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:58:28.280800    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f601a5d4-b3fb-4109-b257-b390871c8197-openshift-service-ca\") pod \"maas-api-e2e-api-a-8090dc-78bb959dd8-qzn96\" (UID: \"f601a5d4-b3fb-4109-b257-b390871c8197\") " pod="opendatahub/maas-api-e2e-api-a-8090dc-78bb959dd8-qzn96"
Jul 06 22:58:28.280970 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:58:28.280836    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f601a5d4-b3fb-4109-b257-b390871c8197-maas-api-tls\") pod \"maas-api-e2e-api-a-8090dc-78bb959dd8-qzn96\" (UID: \"f601a5d4-b3fb-4109-b257-b390871c8197\") " pod="opendatahub/maas-api-e2e-api-a-8090dc-78bb959dd8-qzn96"
Jul 06 22:58:28.281066 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:58:28.280996    2563 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-api-a-8090dc-serving-cert: secret "maas-api-e2e-api-a-8090dc-serving-cert" not found
Jul 06 22:58:28.281101 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:58:28.281077    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f601a5d4-b3fb-4109-b257-b390871c8197-maas-api-tls podName:f601a5d4-b3fb-4109-b257-b390871c8197 nodeName:}" failed. No retries permitted until 2026-07-06 22:58:28.781053228 +0000 UTC m=+2074.206099747 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/f601a5d4-b3fb-4109-b257-b390871c8197-maas-api-tls") pod "maas-api-e2e-api-a-8090dc-78bb959dd8-qzn96" (UID: "f601a5d4-b3fb-4109-b257-b390871c8197") : secret "maas-api-e2e-api-a-8090dc-serving-cert" not found
Jul 06 22:58:28.281559 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:58:28.281542    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f601a5d4-b3fb-4109-b257-b390871c8197-openshift-service-ca\") pod \"maas-api-e2e-api-a-8090dc-78bb959dd8-qzn96\" (UID: \"f601a5d4-b3fb-4109-b257-b390871c8197\") " pod="opendatahub/maas-api-e2e-api-a-8090dc-78bb959dd8-qzn96"
Jul 06 22:58:28.289472 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:58:28.289443    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-766j2\" (UniqueName: \"kubernetes.io/projected/f601a5d4-b3fb-4109-b257-b390871c8197-kube-api-access-766j2\") pod \"maas-api-e2e-api-a-8090dc-78bb959dd8-qzn96\" (UID: \"f601a5d4-b3fb-4109-b257-b390871c8197\") " pod="opendatahub/maas-api-e2e-api-a-8090dc-78bb959dd8-qzn96"
Jul 06 22:58:28.785787 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:58:28.785758    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f601a5d4-b3fb-4109-b257-b390871c8197-maas-api-tls\") pod \"maas-api-e2e-api-a-8090dc-78bb959dd8-qzn96\" (UID: \"f601a5d4-b3fb-4109-b257-b390871c8197\") " pod="opendatahub/maas-api-e2e-api-a-8090dc-78bb959dd8-qzn96"
Jul 06 22:58:28.788267 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:58:28.788246    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f601a5d4-b3fb-4109-b257-b390871c8197-maas-api-tls\") pod \"maas-api-e2e-api-a-8090dc-78bb959dd8-qzn96\" (UID: \"f601a5d4-b3fb-4109-b257-b390871c8197\") " pod="opendatahub/maas-api-e2e-api-a-8090dc-78bb959dd8-qzn96"
Jul 06 22:58:28.965999 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:58:28.965967    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-8090dc-78bb959dd8-qzn96"
Jul 06 22:58:29.104937 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:58:29.104915    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-a-8090dc-78bb959dd8-qzn96"]
Jul 06 22:58:29.107069 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:58:29.107043    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf601a5d4_b3fb_4109_b257_b390871c8197.slice/crio-9c8ecd80de11f5ab84975d1c0cc4eff10c38f61baf1d56cca0d1710a175dbcef WatchSource:0}: Error finding container 9c8ecd80de11f5ab84975d1c0cc4eff10c38f61baf1d56cca0d1710a175dbcef: Status 404 returned error can't find the container with id 9c8ecd80de11f5ab84975d1c0cc4eff10c38f61baf1d56cca0d1710a175dbcef
Jul 06 22:58:29.548306 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:58:29.548267    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-8090dc-78bb959dd8-qzn96" event={"ID":"f601a5d4-b3fb-4109-b257-b390871c8197","Type":"ContainerStarted","Data":"9c8ecd80de11f5ab84975d1c0cc4eff10c38f61baf1d56cca0d1710a175dbcef"}
Jul 06 22:58:30.553713 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:58:30.553677    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-8090dc-78bb959dd8-qzn96" event={"ID":"f601a5d4-b3fb-4109-b257-b390871c8197","Type":"ContainerStarted","Data":"2d939abc333430d5ab35672bade3180c999d9b40aef821a076c8d79b4695c125"}
Jul 06 22:58:30.554187 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:58:30.553804    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-api-a-8090dc-78bb959dd8-qzn96"
Jul 06 22:58:30.573019 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:58:30.572951    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-api-a-8090dc-78bb959dd8-qzn96" podStartSLOduration=2.018245113 podStartE2EDuration="2.572938322s" totalImagesPullingTime="554.693209ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:58:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:58:30.571811642 +0000 UTC m=+2075.996858184" watchObservedRunningTime="2026-07-06 22:58:30.572938322 +0000 UTC m=+2075.997984868"
Jul 06 22:58:33.587480 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:58:33.587441    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9q89m"]
Jul 06 22:58:36.474292 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:58:36.474261    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-api-b-da1e1a-59d4f4cbd8-k9pdg"]
Jul 06 22:58:36.478681 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:58:36.478263    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-da1e1a-59d4f4cbd8-k9pdg"
Jul 06 22:58:36.480678 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:58:36.480648    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-api-b-da1e1a-serving-cert\""
Jul 06 22:58:36.487617 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:58:36.487594    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-b-da1e1a-59d4f4cbd8-k9pdg"]
Jul 06 22:58:36.553019 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:58:36.552983    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c1e562ab-d0d6-4b2a-bbc1-3cae081e38b8-openshift-service-ca\") pod \"maas-api-e2e-api-b-da1e1a-59d4f4cbd8-k9pdg\" (UID: \"c1e562ab-d0d6-4b2a-bbc1-3cae081e38b8\") " pod="opendatahub/maas-api-e2e-api-b-da1e1a-59d4f4cbd8-k9pdg"
Jul 06 22:58:36.553224 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:58:36.553041    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c1e562ab-d0d6-4b2a-bbc1-3cae081e38b8-maas-api-tls\") pod \"maas-api-e2e-api-b-da1e1a-59d4f4cbd8-k9pdg\" (UID: \"c1e562ab-d0d6-4b2a-bbc1-3cae081e38b8\") " pod="opendatahub/maas-api-e2e-api-b-da1e1a-59d4f4cbd8-k9pdg"
Jul 06 22:58:36.553224 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:58:36.553116    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n6r4s\" (UniqueName: \"kubernetes.io/projected/c1e562ab-d0d6-4b2a-bbc1-3cae081e38b8-kube-api-access-n6r4s\") pod \"maas-api-e2e-api-b-da1e1a-59d4f4cbd8-k9pdg\" (UID: \"c1e562ab-d0d6-4b2a-bbc1-3cae081e38b8\") " pod="opendatahub/maas-api-e2e-api-b-da1e1a-59d4f4cbd8-k9pdg"
Jul 06 22:58:36.563344 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:58:36.563321    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-api-a-8090dc-78bb959dd8-qzn96"
Jul 06 22:58:36.654369 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:58:36.654328    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c1e562ab-d0d6-4b2a-bbc1-3cae081e38b8-openshift-service-ca\") pod \"maas-api-e2e-api-b-da1e1a-59d4f4cbd8-k9pdg\" (UID: \"c1e562ab-d0d6-4b2a-bbc1-3cae081e38b8\") " pod="opendatahub/maas-api-e2e-api-b-da1e1a-59d4f4cbd8-k9pdg"
Jul 06 22:58:36.654620 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:58:36.654400    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c1e562ab-d0d6-4b2a-bbc1-3cae081e38b8-maas-api-tls\") pod \"maas-api-e2e-api-b-da1e1a-59d4f4cbd8-k9pdg\" (UID: \"c1e562ab-d0d6-4b2a-bbc1-3cae081e38b8\") " pod="opendatahub/maas-api-e2e-api-b-da1e1a-59d4f4cbd8-k9pdg"
Jul 06 22:58:36.654620 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:58:36.654457    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n6r4s\" (UniqueName: \"kubernetes.io/projected/c1e562ab-d0d6-4b2a-bbc1-3cae081e38b8-kube-api-access-n6r4s\") pod \"maas-api-e2e-api-b-da1e1a-59d4f4cbd8-k9pdg\" (UID: \"c1e562ab-d0d6-4b2a-bbc1-3cae081e38b8\") " pod="opendatahub/maas-api-e2e-api-b-da1e1a-59d4f4cbd8-k9pdg"
Jul 06 22:58:36.654620 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:58:36.654525    2563 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-api-b-da1e1a-serving-cert: secret "maas-api-e2e-api-b-da1e1a-serving-cert" not found
Jul 06 22:58:36.654620 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:58:36.654599    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c1e562ab-d0d6-4b2a-bbc1-3cae081e38b8-maas-api-tls podName:c1e562ab-d0d6-4b2a-bbc1-3cae081e38b8 nodeName:}" failed. No retries permitted until 2026-07-06 22:58:37.154578287 +0000 UTC m=+2082.579624820 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/c1e562ab-d0d6-4b2a-bbc1-3cae081e38b8-maas-api-tls") pod "maas-api-e2e-api-b-da1e1a-59d4f4cbd8-k9pdg" (UID: "c1e562ab-d0d6-4b2a-bbc1-3cae081e38b8") : secret "maas-api-e2e-api-b-da1e1a-serving-cert" not found
Jul 06 22:58:36.654968 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:58:36.654941    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c1e562ab-d0d6-4b2a-bbc1-3cae081e38b8-openshift-service-ca\") pod \"maas-api-e2e-api-b-da1e1a-59d4f4cbd8-k9pdg\" (UID: \"c1e562ab-d0d6-4b2a-bbc1-3cae081e38b8\") " pod="opendatahub/maas-api-e2e-api-b-da1e1a-59d4f4cbd8-k9pdg"
Jul 06 22:58:36.663289 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:58:36.663264    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n6r4s\" (UniqueName: \"kubernetes.io/projected/c1e562ab-d0d6-4b2a-bbc1-3cae081e38b8-kube-api-access-n6r4s\") pod \"maas-api-e2e-api-b-da1e1a-59d4f4cbd8-k9pdg\" (UID: \"c1e562ab-d0d6-4b2a-bbc1-3cae081e38b8\") " pod="opendatahub/maas-api-e2e-api-b-da1e1a-59d4f4cbd8-k9pdg"
Jul 06 22:58:37.157914 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:58:37.157884    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c1e562ab-d0d6-4b2a-bbc1-3cae081e38b8-maas-api-tls\") pod \"maas-api-e2e-api-b-da1e1a-59d4f4cbd8-k9pdg\" (UID: \"c1e562ab-d0d6-4b2a-bbc1-3cae081e38b8\") " pod="opendatahub/maas-api-e2e-api-b-da1e1a-59d4f4cbd8-k9pdg"
Jul 06 22:58:37.160361 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:58:37.160333    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c1e562ab-d0d6-4b2a-bbc1-3cae081e38b8-maas-api-tls\") pod \"maas-api-e2e-api-b-da1e1a-59d4f4cbd8-k9pdg\" (UID: \"c1e562ab-d0d6-4b2a-bbc1-3cae081e38b8\") " pod="opendatahub/maas-api-e2e-api-b-da1e1a-59d4f4cbd8-k9pdg"
Jul 06 22:58:37.392453 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:58:37.392412    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-da1e1a-59d4f4cbd8-k9pdg"
Jul 06 22:58:37.753564 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:58:37.753539    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-b-da1e1a-59d4f4cbd8-k9pdg"]
Jul 06 22:58:37.755880 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:58:37.755851    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1e562ab_d0d6_4b2a_bbc1_3cae081e38b8.slice/crio-a74534f7bc83756e6ff5797095818ad683f634398403c9335f42a1341268d4f1 WatchSource:0}: Error finding container a74534f7bc83756e6ff5797095818ad683f634398403c9335f42a1341268d4f1: Status 404 returned error can't find the container with id a74534f7bc83756e6ff5797095818ad683f634398403c9335f42a1341268d4f1
Jul 06 22:58:38.595761 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:58:38.595680    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-da1e1a-59d4f4cbd8-k9pdg" event={"ID":"c1e562ab-d0d6-4b2a-bbc1-3cae081e38b8","Type":"ContainerStarted","Data":"24e788ed13d735d6283c58ca149f66fe89086b4b3cf7a057b9e51dfc4103a555"}
Jul 06 22:58:38.595761 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:58:38.595731    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-da1e1a-59d4f4cbd8-k9pdg" event={"ID":"c1e562ab-d0d6-4b2a-bbc1-3cae081e38b8","Type":"ContainerStarted","Data":"a74534f7bc83756e6ff5797095818ad683f634398403c9335f42a1341268d4f1"}
Jul 06 22:58:38.595976 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:58:38.595788    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-api-b-da1e1a-59d4f4cbd8-k9pdg"
Jul 06 22:58:38.613499 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:58:38.613382    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-api-b-da1e1a-59d4f4cbd8-k9pdg" podStartSLOduration=2.134690266 podStartE2EDuration="2.613364474s" totalImagesPullingTime="478.674208ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:58:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:58:38.613123331 +0000 UTC m=+2084.038169873" watchObservedRunningTime="2026-07-06 22:58:38.613364474 +0000 UTC m=+2084.038411017"
Jul 06 22:58:43.794547 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:58:43.794512    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9q89m"]
Jul 06 22:58:44.605952 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:58:44.605926    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-api-b-da1e1a-59d4f4cbd8-k9pdg"
Jul 06 22:58:49.483601 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:58:49.483567    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-a-8090dc-78bb959dd8-qzn96"]
Jul 06 22:58:49.484190 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:58:49.483801    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-api-a-8090dc-78bb959dd8-qzn96" podUID="f601a5d4-b3fb-4109-b257-b390871c8197" containerName="maas-api" containerID="cri-o://2d939abc333430d5ab35672bade3180c999d9b40aef821a076c8d79b4695c125" gracePeriod=30
Jul 06 22:58:49.655426 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:58:49.655381    2563 generic.go:356] "Generic (PLEG): container finished" podID="f601a5d4-b3fb-4109-b257-b390871c8197" containerID="2d939abc333430d5ab35672bade3180c999d9b40aef821a076c8d79b4695c125" exitCode=0
Jul 06 22:58:49.655574 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:58:49.655431    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-8090dc-78bb959dd8-qzn96" event={"ID":"f601a5d4-b3fb-4109-b257-b390871c8197","Type":"ContainerDied","Data":"2d939abc333430d5ab35672bade3180c999d9b40aef821a076c8d79b4695c125"}
Jul 06 22:58:49.738752 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:58:49.738695    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-8090dc-78bb959dd8-qzn96"
Jul 06 22:58:49.759019 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:58:49.758986    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f601a5d4-b3fb-4109-b257-b390871c8197-maas-api-tls\") pod \"f601a5d4-b3fb-4109-b257-b390871c8197\" (UID: \"f601a5d4-b3fb-4109-b257-b390871c8197\") "
Jul 06 22:58:49.759154 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:58:49.759070    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f601a5d4-b3fb-4109-b257-b390871c8197-openshift-service-ca\") pod \"f601a5d4-b3fb-4109-b257-b390871c8197\" (UID: \"f601a5d4-b3fb-4109-b257-b390871c8197\") "
Jul 06 22:58:49.759154 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:58:49.759103    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-766j2\" (UniqueName: \"kubernetes.io/projected/f601a5d4-b3fb-4109-b257-b390871c8197-kube-api-access-766j2\") pod \"f601a5d4-b3fb-4109-b257-b390871c8197\" (UID: \"f601a5d4-b3fb-4109-b257-b390871c8197\") "
Jul 06 22:58:49.759423 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:58:49.759400    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f601a5d4-b3fb-4109-b257-b390871c8197-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f601a5d4-b3fb-4109-b257-b390871c8197" (UID: "f601a5d4-b3fb-4109-b257-b390871c8197"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 22:58:49.761357 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:58:49.761330    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f601a5d4-b3fb-4109-b257-b390871c8197-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f601a5d4-b3fb-4109-b257-b390871c8197" (UID: "f601a5d4-b3fb-4109-b257-b390871c8197"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 22:58:49.761483 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:58:49.761377    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f601a5d4-b3fb-4109-b257-b390871c8197-kube-api-access-766j2" (OuterVolumeSpecName: "kube-api-access-766j2") pod "f601a5d4-b3fb-4109-b257-b390871c8197" (UID: "f601a5d4-b3fb-4109-b257-b390871c8197"). InnerVolumeSpecName "kube-api-access-766j2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 22:58:49.860002 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:58:49.859962    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f601a5d4-b3fb-4109-b257-b390871c8197-openshift-service-ca\") on node \"ip-10-0-133-105.ec2.internal\" DevicePath \"\""
Jul 06 22:58:49.860002 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:58:49.859995    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-766j2\" (UniqueName: \"kubernetes.io/projected/f601a5d4-b3fb-4109-b257-b390871c8197-kube-api-access-766j2\") on node \"ip-10-0-133-105.ec2.internal\" DevicePath \"\""
Jul 06 22:58:49.860002 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:58:49.860006    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f601a5d4-b3fb-4109-b257-b390871c8197-maas-api-tls\") on node \"ip-10-0-133-105.ec2.internal\" DevicePath \"\""
Jul 06 22:58:50.661609 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:58:50.661581    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-8090dc-78bb959dd8-qzn96"
Jul 06 22:58:50.661609 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:58:50.661604    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-8090dc-78bb959dd8-qzn96" event={"ID":"f601a5d4-b3fb-4109-b257-b390871c8197","Type":"ContainerDied","Data":"9c8ecd80de11f5ab84975d1c0cc4eff10c38f61baf1d56cca0d1710a175dbcef"}
Jul 06 22:58:50.662085 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:58:50.661658    2563 scope.go:117] "RemoveContainer" containerID="2d939abc333430d5ab35672bade3180c999d9b40aef821a076c8d79b4695c125"
Jul 06 22:58:50.681467 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:58:50.681443    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-a-8090dc-78bb959dd8-qzn96"]
Jul 06 22:58:50.684185 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:58:50.684150    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-api-a-8090dc-78bb959dd8-qzn96"]
Jul 06 22:58:51.080256 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:58:51.080223    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f601a5d4-b3fb-4109-b257-b390871c8197" path="/var/lib/kubelet/pods/f601a5d4-b3fb-4109-b257-b390871c8197/volumes"
Jul 06 22:58:55.158475 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:58:55.158449    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-t6jpk_03309f33-ff47-447d-956f-c89bdd4f8054/ovn-acl-logging/0.log"
Jul 06 22:58:55.159642 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:58:55.159627    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-t6jpk_03309f33-ff47-447d-956f-c89bdd4f8054/ovn-acl-logging/0.log"
Jul 06 22:58:55.685408 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:58:55.685373    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9q89m"]
Jul 06 22:58:56.828238 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:58:56.828203    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-b-da1e1a-59d4f4cbd8-k9pdg"]
Jul 06 22:58:56.828679 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:58:56.828497    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-api-b-da1e1a-59d4f4cbd8-k9pdg" podUID="c1e562ab-d0d6-4b2a-bbc1-3cae081e38b8" containerName="maas-api" containerID="cri-o://24e788ed13d735d6283c58ca149f66fe89086b4b3cf7a057b9e51dfc4103a555" gracePeriod=30
Jul 06 22:58:57.086699 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:58:57.086632    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-da1e1a-59d4f4cbd8-k9pdg"
Jul 06 22:58:57.121350 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:58:57.121325    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n6r4s\" (UniqueName: \"kubernetes.io/projected/c1e562ab-d0d6-4b2a-bbc1-3cae081e38b8-kube-api-access-n6r4s\") pod \"c1e562ab-d0d6-4b2a-bbc1-3cae081e38b8\" (UID: \"c1e562ab-d0d6-4b2a-bbc1-3cae081e38b8\") "
Jul 06 22:58:57.121511 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:58:57.121374    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c1e562ab-d0d6-4b2a-bbc1-3cae081e38b8-openshift-service-ca\") pod \"c1e562ab-d0d6-4b2a-bbc1-3cae081e38b8\" (UID: \"c1e562ab-d0d6-4b2a-bbc1-3cae081e38b8\") "
Jul 06 22:58:57.121511 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:58:57.121402    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c1e562ab-d0d6-4b2a-bbc1-3cae081e38b8-maas-api-tls\") pod \"c1e562ab-d0d6-4b2a-bbc1-3cae081e38b8\" (UID: \"c1e562ab-d0d6-4b2a-bbc1-3cae081e38b8\") "
Jul 06 22:58:57.121819 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:58:57.121787    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c1e562ab-d0d6-4b2a-bbc1-3cae081e38b8-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c1e562ab-d0d6-4b2a-bbc1-3cae081e38b8" (UID: "c1e562ab-d0d6-4b2a-bbc1-3cae081e38b8"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 22:58:57.123632 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:58:57.123605    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c1e562ab-d0d6-4b2a-bbc1-3cae081e38b8-kube-api-access-n6r4s" (OuterVolumeSpecName: "kube-api-access-n6r4s") pod "c1e562ab-d0d6-4b2a-bbc1-3cae081e38b8" (UID: "c1e562ab-d0d6-4b2a-bbc1-3cae081e38b8"). InnerVolumeSpecName "kube-api-access-n6r4s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 22:58:57.123804 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:58:57.123788    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c1e562ab-d0d6-4b2a-bbc1-3cae081e38b8-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c1e562ab-d0d6-4b2a-bbc1-3cae081e38b8" (UID: "c1e562ab-d0d6-4b2a-bbc1-3cae081e38b8"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 22:58:57.222473 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:58:57.222442    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c1e562ab-d0d6-4b2a-bbc1-3cae081e38b8-openshift-service-ca\") on node \"ip-10-0-133-105.ec2.internal\" DevicePath \"\""
Jul 06 22:58:57.222473 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:58:57.222467    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c1e562ab-d0d6-4b2a-bbc1-3cae081e38b8-maas-api-tls\") on node \"ip-10-0-133-105.ec2.internal\" DevicePath \"\""
Jul 06 22:58:57.222473 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:58:57.222478    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n6r4s\" (UniqueName: \"kubernetes.io/projected/c1e562ab-d0d6-4b2a-bbc1-3cae081e38b8-kube-api-access-n6r4s\") on node \"ip-10-0-133-105.ec2.internal\" DevicePath \"\""
Jul 06 22:58:57.699330 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:58:57.699297    2563 generic.go:356] "Generic (PLEG): container finished" podID="c1e562ab-d0d6-4b2a-bbc1-3cae081e38b8" containerID="24e788ed13d735d6283c58ca149f66fe89086b4b3cf7a057b9e51dfc4103a555" exitCode=0
Jul 06 22:58:57.699500 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:58:57.699341    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-da1e1a-59d4f4cbd8-k9pdg" event={"ID":"c1e562ab-d0d6-4b2a-bbc1-3cae081e38b8","Type":"ContainerDied","Data":"24e788ed13d735d6283c58ca149f66fe89086b4b3cf7a057b9e51dfc4103a555"}
Jul 06 22:58:57.699500 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:58:57.699386    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-da1e1a-59d4f4cbd8-k9pdg" event={"ID":"c1e562ab-d0d6-4b2a-bbc1-3cae081e38b8","Type":"ContainerDied","Data":"a74534f7bc83756e6ff5797095818ad683f634398403c9335f42a1341268d4f1"}
Jul 06 22:58:57.699500 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:58:57.699415    2563 scope.go:117] "RemoveContainer" containerID="24e788ed13d735d6283c58ca149f66fe89086b4b3cf7a057b9e51dfc4103a555"
Jul 06 22:58:57.699500 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:58:57.699345    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-da1e1a-59d4f4cbd8-k9pdg"
Jul 06 22:58:57.709164 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:58:57.709148    2563 scope.go:117] "RemoveContainer" containerID="24e788ed13d735d6283c58ca149f66fe89086b4b3cf7a057b9e51dfc4103a555"
Jul 06 22:58:57.709525 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:58:57.709502    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"24e788ed13d735d6283c58ca149f66fe89086b4b3cf7a057b9e51dfc4103a555\": container with ID starting with 24e788ed13d735d6283c58ca149f66fe89086b4b3cf7a057b9e51dfc4103a555 not found: ID does not exist" containerID="24e788ed13d735d6283c58ca149f66fe89086b4b3cf7a057b9e51dfc4103a555"
Jul 06 22:58:57.709590 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:58:57.709532    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"24e788ed13d735d6283c58ca149f66fe89086b4b3cf7a057b9e51dfc4103a555"} err="failed to get container status \"24e788ed13d735d6283c58ca149f66fe89086b4b3cf7a057b9e51dfc4103a555\": rpc error: code = NotFound desc = could not find container \"24e788ed13d735d6283c58ca149f66fe89086b4b3cf7a057b9e51dfc4103a555\": container with ID starting with 24e788ed13d735d6283c58ca149f66fe89086b4b3cf7a057b9e51dfc4103a555 not found: ID does not exist"
Jul 06 22:58:57.719850 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:58:57.719819    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-b-da1e1a-59d4f4cbd8-k9pdg"]
Jul 06 22:58:57.723440 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:58:57.723421    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-api-b-da1e1a-59d4f4cbd8-k9pdg"]
Jul 06 22:58:59.081832 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:58:59.081789    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c1e562ab-d0d6-4b2a-bbc1-3cae081e38b8" path="/var/lib/kubelet/pods/c1e562ab-d0d6-4b2a-bbc1-3cae081e38b8/volumes"
Jul 06 22:59:05.774670 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:59:05.774630    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9q89m"]
Jul 06 22:59:07.296789 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:59:07.296758    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-shared-a-c8326f-84b857bdb5-v92rd"]
Jul 06 22:59:07.297144 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:59:07.297122    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c1e562ab-d0d6-4b2a-bbc1-3cae081e38b8" containerName="maas-api"
Jul 06 22:59:07.297144 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:59:07.297133    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="c1e562ab-d0d6-4b2a-bbc1-3cae081e38b8" containerName="maas-api"
Jul 06 22:59:07.297229 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:59:07.297149    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f601a5d4-b3fb-4109-b257-b390871c8197" containerName="maas-api"
Jul 06 22:59:07.297229 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:59:07.297154    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="f601a5d4-b3fb-4109-b257-b390871c8197" containerName="maas-api"
Jul 06 22:59:07.297298 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:59:07.297237    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="f601a5d4-b3fb-4109-b257-b390871c8197" containerName="maas-api"
Jul 06 22:59:07.297298 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:59:07.297245    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="c1e562ab-d0d6-4b2a-bbc1-3cae081e38b8" containerName="maas-api"
Jul 06 22:59:07.300378 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:59:07.300362    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-c8326f-84b857bdb5-v92rd"
Jul 06 22:59:07.302923 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:59:07.302901    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-shared-a-c8326f-serving-cert\""
Jul 06 22:59:07.303083 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:59:07.303062    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-f7f72\""
Jul 06 22:59:07.309195 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:59:07.309156    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-c8326f-84b857bdb5-v92rd"]
Jul 06 22:59:07.413348 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:59:07.413312    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/671fae4f-a3e8-442d-b0c8-15b3e69d290a-maas-api-tls\") pod \"maas-api-e2e-shared-a-c8326f-84b857bdb5-v92rd\" (UID: \"671fae4f-a3e8-442d-b0c8-15b3e69d290a\") " pod="opendatahub/maas-api-e2e-shared-a-c8326f-84b857bdb5-v92rd"
Jul 06 22:59:07.413348 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:59:07.413352    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/671fae4f-a3e8-442d-b0c8-15b3e69d290a-openshift-service-ca\") pod \"maas-api-e2e-shared-a-c8326f-84b857bdb5-v92rd\" (UID: \"671fae4f-a3e8-442d-b0c8-15b3e69d290a\") " pod="opendatahub/maas-api-e2e-shared-a-c8326f-84b857bdb5-v92rd"
Jul 06 22:59:07.413544 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:59:07.413396    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tswld\" (UniqueName: \"kubernetes.io/projected/671fae4f-a3e8-442d-b0c8-15b3e69d290a-kube-api-access-tswld\") pod \"maas-api-e2e-shared-a-c8326f-84b857bdb5-v92rd\" (UID: \"671fae4f-a3e8-442d-b0c8-15b3e69d290a\") " pod="opendatahub/maas-api-e2e-shared-a-c8326f-84b857bdb5-v92rd"
Jul 06 22:59:07.514368 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:59:07.514332    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tswld\" (UniqueName: \"kubernetes.io/projected/671fae4f-a3e8-442d-b0c8-15b3e69d290a-kube-api-access-tswld\") pod \"maas-api-e2e-shared-a-c8326f-84b857bdb5-v92rd\" (UID: \"671fae4f-a3e8-442d-b0c8-15b3e69d290a\") " pod="opendatahub/maas-api-e2e-shared-a-c8326f-84b857bdb5-v92rd"
Jul 06 22:59:07.514584 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:59:07.514425    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/671fae4f-a3e8-442d-b0c8-15b3e69d290a-maas-api-tls\") pod \"maas-api-e2e-shared-a-c8326f-84b857bdb5-v92rd\" (UID: \"671fae4f-a3e8-442d-b0c8-15b3e69d290a\") " pod="opendatahub/maas-api-e2e-shared-a-c8326f-84b857bdb5-v92rd"
Jul 06 22:59:07.514584 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:59:07.514457    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/671fae4f-a3e8-442d-b0c8-15b3e69d290a-openshift-service-ca\") pod \"maas-api-e2e-shared-a-c8326f-84b857bdb5-v92rd\" (UID: \"671fae4f-a3e8-442d-b0c8-15b3e69d290a\") " pod="opendatahub/maas-api-e2e-shared-a-c8326f-84b857bdb5-v92rd"
Jul 06 22:59:07.515203 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:59:07.515145    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/671fae4f-a3e8-442d-b0c8-15b3e69d290a-openshift-service-ca\") pod \"maas-api-e2e-shared-a-c8326f-84b857bdb5-v92rd\" (UID: \"671fae4f-a3e8-442d-b0c8-15b3e69d290a\") " pod="opendatahub/maas-api-e2e-shared-a-c8326f-84b857bdb5-v92rd"
Jul 06 22:59:07.517268 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:59:07.517243    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/671fae4f-a3e8-442d-b0c8-15b3e69d290a-maas-api-tls\") pod \"maas-api-e2e-shared-a-c8326f-84b857bdb5-v92rd\" (UID: \"671fae4f-a3e8-442d-b0c8-15b3e69d290a\") " pod="opendatahub/maas-api-e2e-shared-a-c8326f-84b857bdb5-v92rd"
Jul 06 22:59:07.522695 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:59:07.522672    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tswld\" (UniqueName: \"kubernetes.io/projected/671fae4f-a3e8-442d-b0c8-15b3e69d290a-kube-api-access-tswld\") pod \"maas-api-e2e-shared-a-c8326f-84b857bdb5-v92rd\" (UID: \"671fae4f-a3e8-442d-b0c8-15b3e69d290a\") " pod="opendatahub/maas-api-e2e-shared-a-c8326f-84b857bdb5-v92rd"
Jul 06 22:59:07.612350 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:59:07.612275    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-c8326f-84b857bdb5-v92rd"
Jul 06 22:59:07.752444 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:59:07.752412    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-c8326f-84b857bdb5-v92rd"]
Jul 06 22:59:07.755799 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:59:07.755759    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod671fae4f_a3e8_442d_b0c8_15b3e69d290a.slice/crio-db65f49a2e9ae1834ec4654d62e0a863debf5468f01e3e725b6008a9f9fdc694 WatchSource:0}: Error finding container db65f49a2e9ae1834ec4654d62e0a863debf5468f01e3e725b6008a9f9fdc694: Status 404 returned error can't find the container with id db65f49a2e9ae1834ec4654d62e0a863debf5468f01e3e725b6008a9f9fdc694
Jul 06 22:59:08.761015 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:59:08.760975    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-c8326f-84b857bdb5-v92rd" event={"ID":"671fae4f-a3e8-442d-b0c8-15b3e69d290a","Type":"ContainerStarted","Data":"6c0122fab9e07a0921af8fcf274bf2f4a4d9dc59a7ee63bdc8986eacc5f1ecee"}
Jul 06 22:59:08.761417 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:59:08.761086    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-c8326f-84b857bdb5-v92rd" event={"ID":"671fae4f-a3e8-442d-b0c8-15b3e69d290a","Type":"ContainerStarted","Data":"db65f49a2e9ae1834ec4654d62e0a863debf5468f01e3e725b6008a9f9fdc694"}
Jul 06 22:59:08.761417 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:59:08.761124    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-shared-a-c8326f-84b857bdb5-v92rd"
Jul 06 22:59:08.778288 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:59:08.778245    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-shared-a-c8326f-84b857bdb5-v92rd" podStartSLOduration=1.469514593 podStartE2EDuration="1.778230947s" totalImagesPullingTime="308.716354ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:59:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:59:08.776478854 +0000 UTC m=+2114.201525395" watchObservedRunningTime="2026-07-06 22:59:08.778230947 +0000 UTC m=+2114.203277487"
Jul 06 22:59:14.771541 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:59:14.771513    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-shared-a-c8326f-84b857bdb5-v92rd"
Jul 06 22:59:15.257251 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:59:15.257220    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-shared-b-b0de62-6df4b6c89-c5h58"]
Jul 06 22:59:15.260895 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:59:15.260870    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-b0de62-6df4b6c89-c5h58"
Jul 06 22:59:15.263728 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:59:15.263704    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-shared-b-b0de62-serving-cert\""
Jul 06 22:59:15.271314 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:59:15.271250    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-b0de62-6df4b6c89-c5h58"]
Jul 06 22:59:15.386142 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:59:15.386110    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-98n8n\" (UniqueName: \"kubernetes.io/projected/5a3baaa4-1dee-40fb-93ae-a6e93dccd2c5-kube-api-access-98n8n\") pod \"maas-api-e2e-shared-b-b0de62-6df4b6c89-c5h58\" (UID: \"5a3baaa4-1dee-40fb-93ae-a6e93dccd2c5\") " pod="opendatahub/maas-api-e2e-shared-b-b0de62-6df4b6c89-c5h58"
Jul 06 22:59:15.386306 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:59:15.386155    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5a3baaa4-1dee-40fb-93ae-a6e93dccd2c5-maas-api-tls\") pod \"maas-api-e2e-shared-b-b0de62-6df4b6c89-c5h58\" (UID: \"5a3baaa4-1dee-40fb-93ae-a6e93dccd2c5\") " pod="opendatahub/maas-api-e2e-shared-b-b0de62-6df4b6c89-c5h58"
Jul 06 22:59:15.386306 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:59:15.386253    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5a3baaa4-1dee-40fb-93ae-a6e93dccd2c5-openshift-service-ca\") pod \"maas-api-e2e-shared-b-b0de62-6df4b6c89-c5h58\" (UID: \"5a3baaa4-1dee-40fb-93ae-a6e93dccd2c5\") " pod="opendatahub/maas-api-e2e-shared-b-b0de62-6df4b6c89-c5h58"
Jul 06 22:59:15.486931 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:59:15.486893    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5a3baaa4-1dee-40fb-93ae-a6e93dccd2c5-maas-api-tls\") pod \"maas-api-e2e-shared-b-b0de62-6df4b6c89-c5h58\" (UID: \"5a3baaa4-1dee-40fb-93ae-a6e93dccd2c5\") " pod="opendatahub/maas-api-e2e-shared-b-b0de62-6df4b6c89-c5h58"
Jul 06 22:59:15.487108 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:59:15.486964    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5a3baaa4-1dee-40fb-93ae-a6e93dccd2c5-openshift-service-ca\") pod \"maas-api-e2e-shared-b-b0de62-6df4b6c89-c5h58\" (UID: \"5a3baaa4-1dee-40fb-93ae-a6e93dccd2c5\") " pod="opendatahub/maas-api-e2e-shared-b-b0de62-6df4b6c89-c5h58"
Jul 06 22:59:15.487108 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:59:15.487013    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-98n8n\" (UniqueName: \"kubernetes.io/projected/5a3baaa4-1dee-40fb-93ae-a6e93dccd2c5-kube-api-access-98n8n\") pod \"maas-api-e2e-shared-b-b0de62-6df4b6c89-c5h58\" (UID: \"5a3baaa4-1dee-40fb-93ae-a6e93dccd2c5\") " pod="opendatahub/maas-api-e2e-shared-b-b0de62-6df4b6c89-c5h58"
Jul 06 22:59:15.487108 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:59:15.487049    2563 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-shared-b-b0de62-serving-cert: secret "maas-api-e2e-shared-b-b0de62-serving-cert" not found
Jul 06 22:59:15.487259 ip-10-0-133-105 kubenswrapper[2563]: E0706 22:59:15.487110    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5a3baaa4-1dee-40fb-93ae-a6e93dccd2c5-maas-api-tls podName:5a3baaa4-1dee-40fb-93ae-a6e93dccd2c5 nodeName:}" failed. No retries permitted until 2026-07-06 22:59:15.987092634 +0000 UTC m=+2121.412139155 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/5a3baaa4-1dee-40fb-93ae-a6e93dccd2c5-maas-api-tls") pod "maas-api-e2e-shared-b-b0de62-6df4b6c89-c5h58" (UID: "5a3baaa4-1dee-40fb-93ae-a6e93dccd2c5") : secret "maas-api-e2e-shared-b-b0de62-serving-cert" not found
Jul 06 22:59:15.487680 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:59:15.487661    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5a3baaa4-1dee-40fb-93ae-a6e93dccd2c5-openshift-service-ca\") pod \"maas-api-e2e-shared-b-b0de62-6df4b6c89-c5h58\" (UID: \"5a3baaa4-1dee-40fb-93ae-a6e93dccd2c5\") " pod="opendatahub/maas-api-e2e-shared-b-b0de62-6df4b6c89-c5h58"
Jul 06 22:59:15.495267 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:59:15.495241    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-98n8n\" (UniqueName: \"kubernetes.io/projected/5a3baaa4-1dee-40fb-93ae-a6e93dccd2c5-kube-api-access-98n8n\") pod \"maas-api-e2e-shared-b-b0de62-6df4b6c89-c5h58\" (UID: \"5a3baaa4-1dee-40fb-93ae-a6e93dccd2c5\") " pod="opendatahub/maas-api-e2e-shared-b-b0de62-6df4b6c89-c5h58"
Jul 06 22:59:15.991324 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:59:15.991287    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5a3baaa4-1dee-40fb-93ae-a6e93dccd2c5-maas-api-tls\") pod \"maas-api-e2e-shared-b-b0de62-6df4b6c89-c5h58\" (UID: \"5a3baaa4-1dee-40fb-93ae-a6e93dccd2c5\") " pod="opendatahub/maas-api-e2e-shared-b-b0de62-6df4b6c89-c5h58"
Jul 06 22:59:15.994102 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:59:15.994076    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5a3baaa4-1dee-40fb-93ae-a6e93dccd2c5-maas-api-tls\") pod \"maas-api-e2e-shared-b-b0de62-6df4b6c89-c5h58\" (UID: \"5a3baaa4-1dee-40fb-93ae-a6e93dccd2c5\") " pod="opendatahub/maas-api-e2e-shared-b-b0de62-6df4b6c89-c5h58"
Jul 06 22:59:16.175315 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:59:16.175278    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-b0de62-6df4b6c89-c5h58"
Jul 06 22:59:16.176248 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:59:16.176221    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9q89m"]
Jul 06 22:59:16.321971 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:59:16.321940    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-b0de62-6df4b6c89-c5h58"]
Jul 06 22:59:16.323534 ip-10-0-133-105 kubenswrapper[2563]: W0706 22:59:16.323491    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a3baaa4_1dee_40fb_93ae_a6e93dccd2c5.slice/crio-ce4562c56efdc96f1309f5d0fb90f22562833d5046abf2b06ebe202eb34a0f30 WatchSource:0}: Error finding container ce4562c56efdc96f1309f5d0fb90f22562833d5046abf2b06ebe202eb34a0f30: Status 404 returned error can't find the container with id ce4562c56efdc96f1309f5d0fb90f22562833d5046abf2b06ebe202eb34a0f30
Jul 06 22:59:16.800104 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:59:16.800075    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-b0de62-6df4b6c89-c5h58" event={"ID":"5a3baaa4-1dee-40fb-93ae-a6e93dccd2c5","Type":"ContainerStarted","Data":"175900dd1bf5f04541fb0bac3bb59a549bc54d386c53cc5c51720547a475eda5"}
Jul 06 22:59:16.800244 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:59:16.800128    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-b0de62-6df4b6c89-c5h58" event={"ID":"5a3baaa4-1dee-40fb-93ae-a6e93dccd2c5","Type":"ContainerStarted","Data":"ce4562c56efdc96f1309f5d0fb90f22562833d5046abf2b06ebe202eb34a0f30"}
Jul 06 22:59:16.800244 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:59:16.800164    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-shared-b-b0de62-6df4b6c89-c5h58"
Jul 06 22:59:16.818101 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:59:16.818044    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-shared-b-b0de62-6df4b6c89-c5h58" podStartSLOduration=1.4457196909999999 podStartE2EDuration="1.818024808s" totalImagesPullingTime="372.305117ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:59:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:59:16.814781532 +0000 UTC m=+2122.239828074" watchObservedRunningTime="2026-07-06 22:59:16.818024808 +0000 UTC m=+2122.243071349"
Jul 06 22:59:22.810125 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:59:22.810099    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-shared-b-b0de62-6df4b6c89-c5h58"
Jul 06 22:59:26.868653 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:59:26.868619    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9q89m"]
Jul 06 22:59:47.669413 ip-10-0-133-105 kubenswrapper[2563]: I0706 22:59:47.669379    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9q89m"]
Jul 06 23:00:07.085555 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:00:07.085518    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9q89m"]
Jul 06 23:00:31.175659 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:00:31.175623    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9q89m"]
Jul 06 23:00:45.176029 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:00:45.175993    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9q89m"]
Jul 06 23:00:57.572380 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:00:57.572345    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9q89m"]
Jul 06 23:01:11.270261 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:01:11.270223    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9q89m"]
Jul 06 23:01:25.576423 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:01:25.576331    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9q89m"]
Jul 06 23:01:39.370086 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:01:39.370041    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9q89m"]
Jul 06 23:01:56.273464 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:01:56.273431    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9q89m"]
Jul 06 23:02:12.274297 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:02:12.274260    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9q89m"]
Jul 06 23:02:13.881241 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:02:13.881201    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-c8326f/sub-test-model-c8326f-kserve-78dd69cf55-nznnq"]
Jul 06 23:02:13.885192 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:02:13.885158    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-c8326f/sub-test-model-c8326f-kserve-78dd69cf55-nznnq"
Jul 06 23:02:13.887620 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:02:13.887590    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-c8326f\"/\"sub-test-model-c8326f-kserve-self-signed-certs\""
Jul 06 23:02:13.888548 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:02:13.888525    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-c8326f\"/\"kube-root-ca.crt\""
Jul 06 23:02:13.889261 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:02:13.888850    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-c8326f\"/\"default-dockercfg-j5m9j\""
Jul 06 23:02:13.889261 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:02:13.888863    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-c8326f\"/\"openshift-service-ca.crt\""
Jul 06 23:02:13.891424 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:02:13.891404    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-c8326f/sub-test-model-c8326f-kserve-78dd69cf55-nznnq"]
Jul 06 23:02:13.988320 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:02:13.988277    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f61d28ad-1c74-47cc-b298-e66b11942cf9-model-cache\") pod \"sub-test-model-c8326f-kserve-78dd69cf55-nznnq\" (UID: \"f61d28ad-1c74-47cc-b298-e66b11942cf9\") " pod="ai-tenant-e2e-shared-a-c8326f/sub-test-model-c8326f-kserve-78dd69cf55-nznnq"
Jul 06 23:02:13.988320 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:02:13.988319    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f61d28ad-1c74-47cc-b298-e66b11942cf9-tmp-dir\") pod \"sub-test-model-c8326f-kserve-78dd69cf55-nznnq\" (UID: \"f61d28ad-1c74-47cc-b298-e66b11942cf9\") " pod="ai-tenant-e2e-shared-a-c8326f/sub-test-model-c8326f-kserve-78dd69cf55-nznnq"
Jul 06 23:02:13.988587 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:02:13.988369    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f61d28ad-1c74-47cc-b298-e66b11942cf9-home\") pod \"sub-test-model-c8326f-kserve-78dd69cf55-nznnq\" (UID: \"f61d28ad-1c74-47cc-b298-e66b11942cf9\") " pod="ai-tenant-e2e-shared-a-c8326f/sub-test-model-c8326f-kserve-78dd69cf55-nznnq"
Jul 06 23:02:13.988587 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:02:13.988438    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f61d28ad-1c74-47cc-b298-e66b11942cf9-dshm\") pod \"sub-test-model-c8326f-kserve-78dd69cf55-nznnq\" (UID: \"f61d28ad-1c74-47cc-b298-e66b11942cf9\") " pod="ai-tenant-e2e-shared-a-c8326f/sub-test-model-c8326f-kserve-78dd69cf55-nznnq"
Jul 06 23:02:13.988587 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:02:13.988468    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4t4zc\" (UniqueName: \"kubernetes.io/projected/f61d28ad-1c74-47cc-b298-e66b11942cf9-kube-api-access-4t4zc\") pod \"sub-test-model-c8326f-kserve-78dd69cf55-nznnq\" (UID: \"f61d28ad-1c74-47cc-b298-e66b11942cf9\") " pod="ai-tenant-e2e-shared-a-c8326f/sub-test-model-c8326f-kserve-78dd69cf55-nznnq"
Jul 06 23:02:13.988587 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:02:13.988528    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f61d28ad-1c74-47cc-b298-e66b11942cf9-tls-certs\") pod \"sub-test-model-c8326f-kserve-78dd69cf55-nznnq\" (UID: \"f61d28ad-1c74-47cc-b298-e66b11942cf9\") " pod="ai-tenant-e2e-shared-a-c8326f/sub-test-model-c8326f-kserve-78dd69cf55-nznnq"
Jul 06 23:02:14.089862 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:02:14.089828    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f61d28ad-1c74-47cc-b298-e66b11942cf9-model-cache\") pod \"sub-test-model-c8326f-kserve-78dd69cf55-nznnq\" (UID: \"f61d28ad-1c74-47cc-b298-e66b11942cf9\") " pod="ai-tenant-e2e-shared-a-c8326f/sub-test-model-c8326f-kserve-78dd69cf55-nznnq"
Jul 06 23:02:14.090029 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:02:14.089870    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f61d28ad-1c74-47cc-b298-e66b11942cf9-tmp-dir\") pod \"sub-test-model-c8326f-kserve-78dd69cf55-nznnq\" (UID: \"f61d28ad-1c74-47cc-b298-e66b11942cf9\") " pod="ai-tenant-e2e-shared-a-c8326f/sub-test-model-c8326f-kserve-78dd69cf55-nznnq"
Jul 06 23:02:14.090029 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:02:14.090009    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f61d28ad-1c74-47cc-b298-e66b11942cf9-home\") pod \"sub-test-model-c8326f-kserve-78dd69cf55-nznnq\" (UID: \"f61d28ad-1c74-47cc-b298-e66b11942cf9\") " pod="ai-tenant-e2e-shared-a-c8326f/sub-test-model-c8326f-kserve-78dd69cf55-nznnq"
Jul 06 23:02:14.090153 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:02:14.090062    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f61d28ad-1c74-47cc-b298-e66b11942cf9-dshm\") pod \"sub-test-model-c8326f-kserve-78dd69cf55-nznnq\" (UID: \"f61d28ad-1c74-47cc-b298-e66b11942cf9\") " pod="ai-tenant-e2e-shared-a-c8326f/sub-test-model-c8326f-kserve-78dd69cf55-nznnq"
Jul 06 23:02:14.090153 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:02:14.090096    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4t4zc\" (UniqueName: \"kubernetes.io/projected/f61d28ad-1c74-47cc-b298-e66b11942cf9-kube-api-access-4t4zc\") pod \"sub-test-model-c8326f-kserve-78dd69cf55-nznnq\" (UID: \"f61d28ad-1c74-47cc-b298-e66b11942cf9\") " pod="ai-tenant-e2e-shared-a-c8326f/sub-test-model-c8326f-kserve-78dd69cf55-nznnq"
Jul 06 23:02:14.090296 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:02:14.090161    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f61d28ad-1c74-47cc-b298-e66b11942cf9-tls-certs\") pod \"sub-test-model-c8326f-kserve-78dd69cf55-nznnq\" (UID: \"f61d28ad-1c74-47cc-b298-e66b11942cf9\") " pod="ai-tenant-e2e-shared-a-c8326f/sub-test-model-c8326f-kserve-78dd69cf55-nznnq"
Jul 06 23:02:14.090350 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:02:14.090293    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f61d28ad-1c74-47cc-b298-e66b11942cf9-model-cache\") pod \"sub-test-model-c8326f-kserve-78dd69cf55-nznnq\" (UID: \"f61d28ad-1c74-47cc-b298-e66b11942cf9\") " pod="ai-tenant-e2e-shared-a-c8326f/sub-test-model-c8326f-kserve-78dd69cf55-nznnq"
Jul 06 23:02:14.090350 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:02:14.090328    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f61d28ad-1c74-47cc-b298-e66b11942cf9-tmp-dir\") pod \"sub-test-model-c8326f-kserve-78dd69cf55-nznnq\" (UID: \"f61d28ad-1c74-47cc-b298-e66b11942cf9\") " pod="ai-tenant-e2e-shared-a-c8326f/sub-test-model-c8326f-kserve-78dd69cf55-nznnq"
Jul 06 23:02:14.090450 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:02:14.090386    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f61d28ad-1c74-47cc-b298-e66b11942cf9-home\") pod \"sub-test-model-c8326f-kserve-78dd69cf55-nznnq\" (UID: \"f61d28ad-1c74-47cc-b298-e66b11942cf9\") " pod="ai-tenant-e2e-shared-a-c8326f/sub-test-model-c8326f-kserve-78dd69cf55-nznnq"
Jul 06 23:02:14.092394 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:02:14.092367    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f61d28ad-1c74-47cc-b298-e66b11942cf9-dshm\") pod \"sub-test-model-c8326f-kserve-78dd69cf55-nznnq\" (UID: \"f61d28ad-1c74-47cc-b298-e66b11942cf9\") " pod="ai-tenant-e2e-shared-a-c8326f/sub-test-model-c8326f-kserve-78dd69cf55-nznnq"
Jul 06 23:02:14.092842 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:02:14.092825    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f61d28ad-1c74-47cc-b298-e66b11942cf9-tls-certs\") pod \"sub-test-model-c8326f-kserve-78dd69cf55-nznnq\" (UID: \"f61d28ad-1c74-47cc-b298-e66b11942cf9\") " pod="ai-tenant-e2e-shared-a-c8326f/sub-test-model-c8326f-kserve-78dd69cf55-nznnq"
Jul 06 23:02:14.097669 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:02:14.097647    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4t4zc\" (UniqueName: \"kubernetes.io/projected/f61d28ad-1c74-47cc-b298-e66b11942cf9-kube-api-access-4t4zc\") pod \"sub-test-model-c8326f-kserve-78dd69cf55-nznnq\" (UID: \"f61d28ad-1c74-47cc-b298-e66b11942cf9\") " pod="ai-tenant-e2e-shared-a-c8326f/sub-test-model-c8326f-kserve-78dd69cf55-nznnq"
Jul 06 23:02:14.203867 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:02:14.203838    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-c8326f/sub-test-model-c8326f-kserve-78dd69cf55-nznnq"
Jul 06 23:02:14.345784 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:02:14.345761    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-c8326f/sub-test-model-c8326f-kserve-78dd69cf55-nznnq"]
Jul 06 23:02:14.347576 ip-10-0-133-105 kubenswrapper[2563]: W0706 23:02:14.347547    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf61d28ad_1c74_47cc_b298_e66b11942cf9.slice/crio-c52bfb8eefa6fa65d6036bbcded5fd697cc4ce4870177d61b0e17594a0a3f02b WatchSource:0}: Error finding container c52bfb8eefa6fa65d6036bbcded5fd697cc4ce4870177d61b0e17594a0a3f02b: Status 404 returned error can't find the container with id c52bfb8eefa6fa65d6036bbcded5fd697cc4ce4870177d61b0e17594a0a3f02b
Jul 06 23:02:14.645721 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:02:14.645621    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-c8326f/sub-test-model-c8326f-kserve-78dd69cf55-nznnq" event={"ID":"f61d28ad-1c74-47cc-b298-e66b11942cf9","Type":"ContainerStarted","Data":"15f6638ad9b3d0fdca6a8509c4b4989e4f81a73f59d6762d78cc5637cb8be697"}
Jul 06 23:02:14.645721 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:02:14.645662    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-c8326f/sub-test-model-c8326f-kserve-78dd69cf55-nznnq" event={"ID":"f61d28ad-1c74-47cc-b298-e66b11942cf9","Type":"ContainerStarted","Data":"c52bfb8eefa6fa65d6036bbcded5fd697cc4ce4870177d61b0e17594a0a3f02b"}
Jul 06 23:02:14.662920 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:02:14.662859    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-c8326f/sub-test-model-c8326f-kserve-78dd69cf55-nznnq" podStartSLOduration=1.662843451 podStartE2EDuration="1.662843451s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:02:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:02:14.660796556 +0000 UTC m=+2300.085843098" watchObservedRunningTime="2026-07-06 23:02:14.662843451 +0000 UTC m=+2300.087889993"
Jul 06 23:02:24.204261 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:02:24.204226    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-c8326f/sub-test-model-c8326f-kserve-78dd69cf55-nznnq"
Jul 06 23:02:24.204810 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:02:24.204275    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-c8326f/sub-test-model-c8326f-kserve-78dd69cf55-nznnq"
Jul 06 23:02:24.217836 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:02:24.217807    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-c8326f/sub-test-model-c8326f-kserve-78dd69cf55-nznnq"
Jul 06 23:02:24.704567 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:02:24.704532    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-c8326f/sub-test-model-c8326f-kserve-78dd69cf55-nznnq"
Jul 06 23:02:27.471677 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:02:27.471640    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9q89m"]
Jul 06 23:02:43.968017 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:02:43.967979    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9q89m"]
Jul 06 23:02:56.279596 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:02:56.279510    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9q89m"]
Jul 06 23:03:11.573067 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:03:11.573032    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-c8326f/sub-test-model-c8326f-kserve-78dd69cf55-nznnq"]
Jul 06 23:03:16.874387 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:03:16.874352    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9q89m"]
Jul 06 23:03:26.594005 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:03:26.593947    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-c8326f/sub-test-model-c8326f-kserve-78dd69cf55-nznnq" podUID="f61d28ad-1c74-47cc-b298-e66b11942cf9" containerName="main" containerID="cri-o://15f6638ad9b3d0fdca6a8509c4b4989e4f81a73f59d6762d78cc5637cb8be697" gracePeriod=45
Jul 06 23:03:26.852854 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:03:26.852792    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-c8326f/sub-test-model-c8326f-kserve-78dd69cf55-nznnq"
Jul 06 23:03:26.950987 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:03:26.950947    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f61d28ad-1c74-47cc-b298-e66b11942cf9-model-cache\") pod \"f61d28ad-1c74-47cc-b298-e66b11942cf9\" (UID: \"f61d28ad-1c74-47cc-b298-e66b11942cf9\") "
Jul 06 23:03:26.950987 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:03:26.950986    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f61d28ad-1c74-47cc-b298-e66b11942cf9-dshm\") pod \"f61d28ad-1c74-47cc-b298-e66b11942cf9\" (UID: \"f61d28ad-1c74-47cc-b298-e66b11942cf9\") "
Jul 06 23:03:26.951246 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:03:26.951010    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f61d28ad-1c74-47cc-b298-e66b11942cf9-home\") pod \"f61d28ad-1c74-47cc-b298-e66b11942cf9\" (UID: \"f61d28ad-1c74-47cc-b298-e66b11942cf9\") "
Jul 06 23:03:26.951246 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:03:26.951089    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f61d28ad-1c74-47cc-b298-e66b11942cf9-tls-certs\") pod \"f61d28ad-1c74-47cc-b298-e66b11942cf9\" (UID: \"f61d28ad-1c74-47cc-b298-e66b11942cf9\") "
Jul 06 23:03:26.951246 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:03:26.951159    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f61d28ad-1c74-47cc-b298-e66b11942cf9-tmp-dir\") pod \"f61d28ad-1c74-47cc-b298-e66b11942cf9\" (UID: \"f61d28ad-1c74-47cc-b298-e66b11942cf9\") "
Jul 06 23:03:26.951246 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:03:26.951215    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4t4zc\" (UniqueName: \"kubernetes.io/projected/f61d28ad-1c74-47cc-b298-e66b11942cf9-kube-api-access-4t4zc\") pod \"f61d28ad-1c74-47cc-b298-e66b11942cf9\" (UID: \"f61d28ad-1c74-47cc-b298-e66b11942cf9\") "
Jul 06 23:03:26.951475 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:03:26.951323    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f61d28ad-1c74-47cc-b298-e66b11942cf9-home" (OuterVolumeSpecName: "home") pod "f61d28ad-1c74-47cc-b298-e66b11942cf9" (UID: "f61d28ad-1c74-47cc-b298-e66b11942cf9"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 23:03:26.951475 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:03:26.951337    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f61d28ad-1c74-47cc-b298-e66b11942cf9-model-cache" (OuterVolumeSpecName: "model-cache") pod "f61d28ad-1c74-47cc-b298-e66b11942cf9" (UID: "f61d28ad-1c74-47cc-b298-e66b11942cf9"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 23:03:26.951585 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:03:26.951501    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f61d28ad-1c74-47cc-b298-e66b11942cf9-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f61d28ad-1c74-47cc-b298-e66b11942cf9" (UID: "f61d28ad-1c74-47cc-b298-e66b11942cf9"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 23:03:26.951585 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:03:26.951528    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f61d28ad-1c74-47cc-b298-e66b11942cf9-model-cache\") on node \"ip-10-0-133-105.ec2.internal\" DevicePath \"\""
Jul 06 23:03:26.951585 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:03:26.951545    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f61d28ad-1c74-47cc-b298-e66b11942cf9-home\") on node \"ip-10-0-133-105.ec2.internal\" DevicePath \"\""
Jul 06 23:03:26.953818 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:03:26.953791    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f61d28ad-1c74-47cc-b298-e66b11942cf9-kube-api-access-4t4zc" (OuterVolumeSpecName: "kube-api-access-4t4zc") pod "f61d28ad-1c74-47cc-b298-e66b11942cf9" (UID: "f61d28ad-1c74-47cc-b298-e66b11942cf9"). InnerVolumeSpecName "kube-api-access-4t4zc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 23:03:26.953958 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:03:26.953940    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f61d28ad-1c74-47cc-b298-e66b11942cf9-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f61d28ad-1c74-47cc-b298-e66b11942cf9" (UID: "f61d28ad-1c74-47cc-b298-e66b11942cf9"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 23:03:26.954011 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:03:26.953960    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f61d28ad-1c74-47cc-b298-e66b11942cf9-dshm" (OuterVolumeSpecName: "dshm") pod "f61d28ad-1c74-47cc-b298-e66b11942cf9" (UID: "f61d28ad-1c74-47cc-b298-e66b11942cf9"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 23:03:26.998227 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:03:26.998197    2563 generic.go:356] "Generic (PLEG): container finished" podID="f61d28ad-1c74-47cc-b298-e66b11942cf9" containerID="15f6638ad9b3d0fdca6a8509c4b4989e4f81a73f59d6762d78cc5637cb8be697" exitCode=0
Jul 06 23:03:26.998362 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:03:26.998234    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-c8326f/sub-test-model-c8326f-kserve-78dd69cf55-nznnq"
Jul 06 23:03:26.998362 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:03:26.998272    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-c8326f/sub-test-model-c8326f-kserve-78dd69cf55-nznnq" event={"ID":"f61d28ad-1c74-47cc-b298-e66b11942cf9","Type":"ContainerDied","Data":"15f6638ad9b3d0fdca6a8509c4b4989e4f81a73f59d6762d78cc5637cb8be697"}
Jul 06 23:03:26.998362 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:03:26.998319    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-c8326f/sub-test-model-c8326f-kserve-78dd69cf55-nznnq" event={"ID":"f61d28ad-1c74-47cc-b298-e66b11942cf9","Type":"ContainerDied","Data":"c52bfb8eefa6fa65d6036bbcded5fd697cc4ce4870177d61b0e17594a0a3f02b"}
Jul 06 23:03:26.998362 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:03:26.998335    2563 scope.go:117] "RemoveContainer" containerID="15f6638ad9b3d0fdca6a8509c4b4989e4f81a73f59d6762d78cc5637cb8be697"
Jul 06 23:03:27.007309 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:03:27.007293    2563 scope.go:117] "RemoveContainer" containerID="15f6638ad9b3d0fdca6a8509c4b4989e4f81a73f59d6762d78cc5637cb8be697"
Jul 06 23:03:27.007557 ip-10-0-133-105 kubenswrapper[2563]: E0706 23:03:27.007539    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"15f6638ad9b3d0fdca6a8509c4b4989e4f81a73f59d6762d78cc5637cb8be697\": container with ID starting with 15f6638ad9b3d0fdca6a8509c4b4989e4f81a73f59d6762d78cc5637cb8be697 not found: ID does not exist" containerID="15f6638ad9b3d0fdca6a8509c4b4989e4f81a73f59d6762d78cc5637cb8be697"
Jul 06 23:03:27.007608 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:03:27.007570    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"15f6638ad9b3d0fdca6a8509c4b4989e4f81a73f59d6762d78cc5637cb8be697"} err="failed to get container status \"15f6638ad9b3d0fdca6a8509c4b4989e4f81a73f59d6762d78cc5637cb8be697\": rpc error: code = NotFound desc = could not find container \"15f6638ad9b3d0fdca6a8509c4b4989e4f81a73f59d6762d78cc5637cb8be697\": container with ID starting with 15f6638ad9b3d0fdca6a8509c4b4989e4f81a73f59d6762d78cc5637cb8be697 not found: ID does not exist"
Jul 06 23:03:27.021313 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:03:27.021290    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-c8326f/sub-test-model-c8326f-kserve-78dd69cf55-nznnq"]
Jul 06 23:03:27.025826 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:03:27.025804    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-c8326f/sub-test-model-c8326f-kserve-78dd69cf55-nznnq"]
Jul 06 23:03:27.052399 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:03:27.052374    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f61d28ad-1c74-47cc-b298-e66b11942cf9-tls-certs\") on node \"ip-10-0-133-105.ec2.internal\" DevicePath \"\""
Jul 06 23:03:27.052399 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:03:27.052395    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f61d28ad-1c74-47cc-b298-e66b11942cf9-tmp-dir\") on node \"ip-10-0-133-105.ec2.internal\" DevicePath \"\""
Jul 06 23:03:27.052541 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:03:27.052404    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4t4zc\" (UniqueName: \"kubernetes.io/projected/f61d28ad-1c74-47cc-b298-e66b11942cf9-kube-api-access-4t4zc\") on node \"ip-10-0-133-105.ec2.internal\" DevicePath \"\""
Jul 06 23:03:27.052541 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:03:27.052415    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f61d28ad-1c74-47cc-b298-e66b11942cf9-dshm\") on node \"ip-10-0-133-105.ec2.internal\" DevicePath \"\""
Jul 06 23:03:27.080186 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:03:27.080141    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f61d28ad-1c74-47cc-b298-e66b11942cf9" path="/var/lib/kubelet/pods/f61d28ad-1c74-47cc-b298-e66b11942cf9/volumes"
Jul 06 23:03:32.681674 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:03:32.681635    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9q89m"]
Jul 06 23:03:37.971453 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:03:37.971334    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-b0de62/rate-test-model-b0de62-kserve-7969fb8bcf-gjkxl"]
Jul 06 23:03:37.971979 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:03:37.971931    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f61d28ad-1c74-47cc-b298-e66b11942cf9" containerName="main"
Jul 06 23:03:37.971979 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:03:37.971956    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="f61d28ad-1c74-47cc-b298-e66b11942cf9" containerName="main"
Jul 06 23:03:37.972150 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:03:37.972053    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="f61d28ad-1c74-47cc-b298-e66b11942cf9" containerName="main"
Jul 06 23:03:37.975532 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:03:37.975512    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-b0de62/rate-test-model-b0de62-kserve-7969fb8bcf-gjkxl"
Jul 06 23:03:37.978314 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:03:37.978293    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-b0de62\"/\"kube-root-ca.crt\""
Jul 06 23:03:37.979355 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:03:37.979336    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-b0de62\"/\"openshift-service-ca.crt\""
Jul 06 23:03:37.979355 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:03:37.979347    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-b0de62\"/\"default-dockercfg-pcr2j\""
Jul 06 23:03:37.979659 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:03:37.979643    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-b0de62\"/\"rate-test-model-b0de62-kserve-self-signed-certs\""
Jul 06 23:03:37.987947 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:03:37.987922    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-b0de62/rate-test-model-b0de62-kserve-7969fb8bcf-gjkxl"]
Jul 06 23:03:38.050027 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:03:38.049998    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/35e3fbe6-ec61-4186-bfcb-481d52cfde31-tls-certs\") pod \"rate-test-model-b0de62-kserve-7969fb8bcf-gjkxl\" (UID: \"35e3fbe6-ec61-4186-bfcb-481d52cfde31\") " pod="ai-tenant-e2e-shared-b-b0de62/rate-test-model-b0de62-kserve-7969fb8bcf-gjkxl"
Jul 06 23:03:38.050027 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:03:38.050034    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/35e3fbe6-ec61-4186-bfcb-481d52cfde31-home\") pod \"rate-test-model-b0de62-kserve-7969fb8bcf-gjkxl\" (UID: \"35e3fbe6-ec61-4186-bfcb-481d52cfde31\") " pod="ai-tenant-e2e-shared-b-b0de62/rate-test-model-b0de62-kserve-7969fb8bcf-gjkxl"
Jul 06 23:03:38.050268 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:03:38.050059    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-764d8\" (UniqueName: \"kubernetes.io/projected/35e3fbe6-ec61-4186-bfcb-481d52cfde31-kube-api-access-764d8\") pod \"rate-test-model-b0de62-kserve-7969fb8bcf-gjkxl\" (UID: \"35e3fbe6-ec61-4186-bfcb-481d52cfde31\") " pod="ai-tenant-e2e-shared-b-b0de62/rate-test-model-b0de62-kserve-7969fb8bcf-gjkxl"
Jul 06 23:03:38.050268 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:03:38.050077    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/35e3fbe6-ec61-4186-bfcb-481d52cfde31-tmp-dir\") pod \"rate-test-model-b0de62-kserve-7969fb8bcf-gjkxl\" (UID: \"35e3fbe6-ec61-4186-bfcb-481d52cfde31\") " pod="ai-tenant-e2e-shared-b-b0de62/rate-test-model-b0de62-kserve-7969fb8bcf-gjkxl"
Jul 06 23:03:38.050268 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:03:38.050143    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/35e3fbe6-ec61-4186-bfcb-481d52cfde31-model-cache\") pod \"rate-test-model-b0de62-kserve-7969fb8bcf-gjkxl\" (UID: \"35e3fbe6-ec61-4186-bfcb-481d52cfde31\") " pod="ai-tenant-e2e-shared-b-b0de62/rate-test-model-b0de62-kserve-7969fb8bcf-gjkxl"
Jul 06 23:03:38.050268 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:03:38.050221    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/35e3fbe6-ec61-4186-bfcb-481d52cfde31-dshm\") pod \"rate-test-model-b0de62-kserve-7969fb8bcf-gjkxl\" (UID: \"35e3fbe6-ec61-4186-bfcb-481d52cfde31\") " pod="ai-tenant-e2e-shared-b-b0de62/rate-test-model-b0de62-kserve-7969fb8bcf-gjkxl"
Jul 06 23:03:38.151356 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:03:38.151321    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/35e3fbe6-ec61-4186-bfcb-481d52cfde31-model-cache\") pod \"rate-test-model-b0de62-kserve-7969fb8bcf-gjkxl\" (UID: \"35e3fbe6-ec61-4186-bfcb-481d52cfde31\") " pod="ai-tenant-e2e-shared-b-b0de62/rate-test-model-b0de62-kserve-7969fb8bcf-gjkxl"
Jul 06 23:03:38.151547 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:03:38.151380    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/35e3fbe6-ec61-4186-bfcb-481d52cfde31-dshm\") pod \"rate-test-model-b0de62-kserve-7969fb8bcf-gjkxl\" (UID: \"35e3fbe6-ec61-4186-bfcb-481d52cfde31\") " pod="ai-tenant-e2e-shared-b-b0de62/rate-test-model-b0de62-kserve-7969fb8bcf-gjkxl"
Jul 06 23:03:38.151547 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:03:38.151443    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/35e3fbe6-ec61-4186-bfcb-481d52cfde31-tls-certs\") pod \"rate-test-model-b0de62-kserve-7969fb8bcf-gjkxl\" (UID: \"35e3fbe6-ec61-4186-bfcb-481d52cfde31\") " pod="ai-tenant-e2e-shared-b-b0de62/rate-test-model-b0de62-kserve-7969fb8bcf-gjkxl"
Jul 06 23:03:38.151547 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:03:38.151472    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/35e3fbe6-ec61-4186-bfcb-481d52cfde31-home\") pod \"rate-test-model-b0de62-kserve-7969fb8bcf-gjkxl\" (UID: \"35e3fbe6-ec61-4186-bfcb-481d52cfde31\") " pod="ai-tenant-e2e-shared-b-b0de62/rate-test-model-b0de62-kserve-7969fb8bcf-gjkxl"
Jul 06 23:03:38.151547 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:03:38.151497    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-764d8\" (UniqueName: \"kubernetes.io/projected/35e3fbe6-ec61-4186-bfcb-481d52cfde31-kube-api-access-764d8\") pod \"rate-test-model-b0de62-kserve-7969fb8bcf-gjkxl\" (UID: \"35e3fbe6-ec61-4186-bfcb-481d52cfde31\") " pod="ai-tenant-e2e-shared-b-b0de62/rate-test-model-b0de62-kserve-7969fb8bcf-gjkxl"
Jul 06 23:03:38.151547 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:03:38.151520    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/35e3fbe6-ec61-4186-bfcb-481d52cfde31-tmp-dir\") pod \"rate-test-model-b0de62-kserve-7969fb8bcf-gjkxl\" (UID: \"35e3fbe6-ec61-4186-bfcb-481d52cfde31\") " pod="ai-tenant-e2e-shared-b-b0de62/rate-test-model-b0de62-kserve-7969fb8bcf-gjkxl"
Jul 06 23:03:38.151810 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:03:38.151790    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/35e3fbe6-ec61-4186-bfcb-481d52cfde31-model-cache\") pod \"rate-test-model-b0de62-kserve-7969fb8bcf-gjkxl\" (UID: \"35e3fbe6-ec61-4186-bfcb-481d52cfde31\") " pod="ai-tenant-e2e-shared-b-b0de62/rate-test-model-b0de62-kserve-7969fb8bcf-gjkxl"
Jul 06 23:03:38.151982 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:03:38.151951    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/35e3fbe6-ec61-4186-bfcb-481d52cfde31-home\") pod \"rate-test-model-b0de62-kserve-7969fb8bcf-gjkxl\" (UID: \"35e3fbe6-ec61-4186-bfcb-481d52cfde31\") " pod="ai-tenant-e2e-shared-b-b0de62/rate-test-model-b0de62-kserve-7969fb8bcf-gjkxl"
Jul 06 23:03:38.152270 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:03:38.152066    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/35e3fbe6-ec61-4186-bfcb-481d52cfde31-tmp-dir\") pod \"rate-test-model-b0de62-kserve-7969fb8bcf-gjkxl\" (UID: \"35e3fbe6-ec61-4186-bfcb-481d52cfde31\") " pod="ai-tenant-e2e-shared-b-b0de62/rate-test-model-b0de62-kserve-7969fb8bcf-gjkxl"
Jul 06 23:03:38.153857 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:03:38.153836    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/35e3fbe6-ec61-4186-bfcb-481d52cfde31-dshm\") pod \"rate-test-model-b0de62-kserve-7969fb8bcf-gjkxl\" (UID: \"35e3fbe6-ec61-4186-bfcb-481d52cfde31\") " pod="ai-tenant-e2e-shared-b-b0de62/rate-test-model-b0de62-kserve-7969fb8bcf-gjkxl"
Jul 06 23:03:38.154031 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:03:38.154013    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/35e3fbe6-ec61-4186-bfcb-481d52cfde31-tls-certs\") pod \"rate-test-model-b0de62-kserve-7969fb8bcf-gjkxl\" (UID: \"35e3fbe6-ec61-4186-bfcb-481d52cfde31\") " pod="ai-tenant-e2e-shared-b-b0de62/rate-test-model-b0de62-kserve-7969fb8bcf-gjkxl"
Jul 06 23:03:38.159282 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:03:38.159264    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-764d8\" (UniqueName: \"kubernetes.io/projected/35e3fbe6-ec61-4186-bfcb-481d52cfde31-kube-api-access-764d8\") pod \"rate-test-model-b0de62-kserve-7969fb8bcf-gjkxl\" (UID: \"35e3fbe6-ec61-4186-bfcb-481d52cfde31\") " pod="ai-tenant-e2e-shared-b-b0de62/rate-test-model-b0de62-kserve-7969fb8bcf-gjkxl"
Jul 06 23:03:38.293605 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:03:38.293515    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-b0de62/rate-test-model-b0de62-kserve-7969fb8bcf-gjkxl"
Jul 06 23:03:38.432673 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:03:38.432646    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-b0de62/rate-test-model-b0de62-kserve-7969fb8bcf-gjkxl"]
Jul 06 23:03:38.435140 ip-10-0-133-105 kubenswrapper[2563]: W0706 23:03:38.435104    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod35e3fbe6_ec61_4186_bfcb_481d52cfde31.slice/crio-56196d7e3ac65f20d3ee3ffee5fb5a5e87a38f9ff6cabe17726aaa98d4b387b5 WatchSource:0}: Error finding container 56196d7e3ac65f20d3ee3ffee5fb5a5e87a38f9ff6cabe17726aaa98d4b387b5: Status 404 returned error can't find the container with id 56196d7e3ac65f20d3ee3ffee5fb5a5e87a38f9ff6cabe17726aaa98d4b387b5
Jul 06 23:03:39.056303 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:03:39.056271    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-b0de62/rate-test-model-b0de62-kserve-7969fb8bcf-gjkxl" event={"ID":"35e3fbe6-ec61-4186-bfcb-481d52cfde31","Type":"ContainerStarted","Data":"bc1fb82c0bc2f56c690bc6123ac3ef1959e572b25d4b4303cc073adc7cdc3674"}
Jul 06 23:03:39.056303 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:03:39.056310    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-b0de62/rate-test-model-b0de62-kserve-7969fb8bcf-gjkxl" event={"ID":"35e3fbe6-ec61-4186-bfcb-481d52cfde31","Type":"ContainerStarted","Data":"56196d7e3ac65f20d3ee3ffee5fb5a5e87a38f9ff6cabe17726aaa98d4b387b5"}
Jul 06 23:03:39.073935 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:03:39.073888    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-b0de62/rate-test-model-b0de62-kserve-7969fb8bcf-gjkxl" podStartSLOduration=2.073873806 podStartE2EDuration="2.073873806s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:03:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:03:39.071903104 +0000 UTC m=+2384.496949647" watchObservedRunningTime="2026-07-06 23:03:39.073873806 +0000 UTC m=+2384.498920393"
Jul 06 23:03:43.074197 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:03:43.074148    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9q89m"]
Jul 06 23:03:48.294017 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:03:48.293984    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-b0de62/rate-test-model-b0de62-kserve-7969fb8bcf-gjkxl"
Jul 06 23:03:48.294639 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:03:48.294311    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-b0de62/rate-test-model-b0de62-kserve-7969fb8bcf-gjkxl"
Jul 06 23:03:48.306486 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:03:48.306465    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-b0de62/rate-test-model-b0de62-kserve-7969fb8bcf-gjkxl"
Jul 06 23:03:49.112543 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:03:49.112517    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-b0de62/rate-test-model-b0de62-kserve-7969fb8bcf-gjkxl"
Jul 06 23:03:55.187973 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:03:55.187934    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-t6jpk_03309f33-ff47-447d-956f-c89bdd4f8054/ovn-acl-logging/0.log"
Jul 06 23:03:55.189698 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:03:55.189676    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-t6jpk_03309f33-ff47-447d-956f-c89bdd4f8054/ovn-acl-logging/0.log"
Jul 06 23:03:56.868396 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:03:56.868361    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9q89m"]
Jul 06 23:04:15.178068 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:04:15.178035    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9q89m"]
Jul 06 23:04:20.293646 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:04:20.293387    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-b0de62/rate-test-model-b0de62-kserve-7969fb8bcf-gjkxl"]
Jul 06 23:04:22.084439 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:04:22.084399    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-c8326f-84b857bdb5-v92rd"]
Jul 06 23:04:22.085283 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:04:22.085146    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-shared-a-c8326f-84b857bdb5-v92rd" podUID="671fae4f-a3e8-442d-b0c8-15b3e69d290a" containerName="maas-api" containerID="cri-o://6c0122fab9e07a0921af8fcf274bf2f4a4d9dc59a7ee63bdc8986eacc5f1ecee" gracePeriod=30
Jul 06 23:04:22.262183 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:04:22.262114    2563 generic.go:356] "Generic (PLEG): container finished" podID="671fae4f-a3e8-442d-b0c8-15b3e69d290a" containerID="6c0122fab9e07a0921af8fcf274bf2f4a4d9dc59a7ee63bdc8986eacc5f1ecee" exitCode=0
Jul 06 23:04:22.262333 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:04:22.262242    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-c8326f-84b857bdb5-v92rd" event={"ID":"671fae4f-a3e8-442d-b0c8-15b3e69d290a","Type":"ContainerDied","Data":"6c0122fab9e07a0921af8fcf274bf2f4a4d9dc59a7ee63bdc8986eacc5f1ecee"}
Jul 06 23:04:22.339832 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:04:22.339768    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-c8326f-84b857bdb5-v92rd"
Jul 06 23:04:22.445909 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:04:22.445878    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/671fae4f-a3e8-442d-b0c8-15b3e69d290a-maas-api-tls\") pod \"671fae4f-a3e8-442d-b0c8-15b3e69d290a\" (UID: \"671fae4f-a3e8-442d-b0c8-15b3e69d290a\") "
Jul 06 23:04:22.445909 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:04:22.445921    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/671fae4f-a3e8-442d-b0c8-15b3e69d290a-openshift-service-ca\") pod \"671fae4f-a3e8-442d-b0c8-15b3e69d290a\" (UID: \"671fae4f-a3e8-442d-b0c8-15b3e69d290a\") "
Jul 06 23:04:22.446215 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:04:22.445958    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tswld\" (UniqueName: \"kubernetes.io/projected/671fae4f-a3e8-442d-b0c8-15b3e69d290a-kube-api-access-tswld\") pod \"671fae4f-a3e8-442d-b0c8-15b3e69d290a\" (UID: \"671fae4f-a3e8-442d-b0c8-15b3e69d290a\") "
Jul 06 23:04:22.446353 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:04:22.446326    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/671fae4f-a3e8-442d-b0c8-15b3e69d290a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "671fae4f-a3e8-442d-b0c8-15b3e69d290a" (UID: "671fae4f-a3e8-442d-b0c8-15b3e69d290a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 23:04:22.448211 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:04:22.448187    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/671fae4f-a3e8-442d-b0c8-15b3e69d290a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "671fae4f-a3e8-442d-b0c8-15b3e69d290a" (UID: "671fae4f-a3e8-442d-b0c8-15b3e69d290a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 23:04:22.448320 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:04:22.448280    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/671fae4f-a3e8-442d-b0c8-15b3e69d290a-kube-api-access-tswld" (OuterVolumeSpecName: "kube-api-access-tswld") pod "671fae4f-a3e8-442d-b0c8-15b3e69d290a" (UID: "671fae4f-a3e8-442d-b0c8-15b3e69d290a"). InnerVolumeSpecName "kube-api-access-tswld". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 23:04:22.547214 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:04:22.547160    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/671fae4f-a3e8-442d-b0c8-15b3e69d290a-maas-api-tls\") on node \"ip-10-0-133-105.ec2.internal\" DevicePath \"\""
Jul 06 23:04:22.547214 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:04:22.547211    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/671fae4f-a3e8-442d-b0c8-15b3e69d290a-openshift-service-ca\") on node \"ip-10-0-133-105.ec2.internal\" DevicePath \"\""
Jul 06 23:04:22.547405 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:04:22.547225    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tswld\" (UniqueName: \"kubernetes.io/projected/671fae4f-a3e8-442d-b0c8-15b3e69d290a-kube-api-access-tswld\") on node \"ip-10-0-133-105.ec2.internal\" DevicePath \"\""
Jul 06 23:04:23.268382 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:04:23.268354    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-c8326f-84b857bdb5-v92rd"
Jul 06 23:04:23.268811 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:04:23.268380    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-c8326f-84b857bdb5-v92rd" event={"ID":"671fae4f-a3e8-442d-b0c8-15b3e69d290a","Type":"ContainerDied","Data":"db65f49a2e9ae1834ec4654d62e0a863debf5468f01e3e725b6008a9f9fdc694"}
Jul 06 23:04:23.268811 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:04:23.268462    2563 scope.go:117] "RemoveContainer" containerID="6c0122fab9e07a0921af8fcf274bf2f4a4d9dc59a7ee63bdc8986eacc5f1ecee"
Jul 06 23:04:23.284306 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:04:23.284278    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-c8326f-84b857bdb5-v92rd"]
Jul 06 23:04:23.287663 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:04:23.287639    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-c8326f-84b857bdb5-v92rd"]
Jul 06 23:04:25.079958 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:04:25.079925    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="671fae4f-a3e8-442d-b0c8-15b3e69d290a" path="/var/lib/kubelet/pods/671fae4f-a3e8-442d-b0c8-15b3e69d290a/volumes"
Jul 06 23:04:30.980488 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:04:30.980449    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9q89m"]
Jul 06 23:04:35.315557 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:04:35.315508    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-b0de62/rate-test-model-b0de62-kserve-7969fb8bcf-gjkxl" podUID="35e3fbe6-ec61-4186-bfcb-481d52cfde31" containerName="main" containerID="cri-o://bc1fb82c0bc2f56c690bc6123ac3ef1959e572b25d4b4303cc073adc7cdc3674" gracePeriod=45
Jul 06 23:04:35.570366 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:04:35.570307    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-b0de62/rate-test-model-b0de62-kserve-7969fb8bcf-gjkxl"
Jul 06 23:04:35.672778 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:04:35.672745    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/35e3fbe6-ec61-4186-bfcb-481d52cfde31-tmp-dir\") pod \"35e3fbe6-ec61-4186-bfcb-481d52cfde31\" (UID: \"35e3fbe6-ec61-4186-bfcb-481d52cfde31\") "
Jul 06 23:04:35.672960 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:04:35.672833    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/35e3fbe6-ec61-4186-bfcb-481d52cfde31-dshm\") pod \"35e3fbe6-ec61-4186-bfcb-481d52cfde31\" (UID: \"35e3fbe6-ec61-4186-bfcb-481d52cfde31\") "
Jul 06 23:04:35.672960 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:04:35.672878    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-764d8\" (UniqueName: \"kubernetes.io/projected/35e3fbe6-ec61-4186-bfcb-481d52cfde31-kube-api-access-764d8\") pod \"35e3fbe6-ec61-4186-bfcb-481d52cfde31\" (UID: \"35e3fbe6-ec61-4186-bfcb-481d52cfde31\") "
Jul 06 23:04:35.672960 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:04:35.672922    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/35e3fbe6-ec61-4186-bfcb-481d52cfde31-home\") pod \"35e3fbe6-ec61-4186-bfcb-481d52cfde31\" (UID: \"35e3fbe6-ec61-4186-bfcb-481d52cfde31\") "
Jul 06 23:04:35.672960 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:04:35.672960    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/35e3fbe6-ec61-4186-bfcb-481d52cfde31-tls-certs\") pod \"35e3fbe6-ec61-4186-bfcb-481d52cfde31\" (UID: \"35e3fbe6-ec61-4186-bfcb-481d52cfde31\") "
Jul 06 23:04:35.673219 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:04:35.672991    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/35e3fbe6-ec61-4186-bfcb-481d52cfde31-model-cache\") pod \"35e3fbe6-ec61-4186-bfcb-481d52cfde31\" (UID: \"35e3fbe6-ec61-4186-bfcb-481d52cfde31\") "
Jul 06 23:04:35.673219 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:04:35.673015    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/35e3fbe6-ec61-4186-bfcb-481d52cfde31-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "35e3fbe6-ec61-4186-bfcb-481d52cfde31" (UID: "35e3fbe6-ec61-4186-bfcb-481d52cfde31"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 23:04:35.673219 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:04:35.673215    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/35e3fbe6-ec61-4186-bfcb-481d52cfde31-tmp-dir\") on node \"ip-10-0-133-105.ec2.internal\" DevicePath \"\""
Jul 06 23:04:35.673376 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:04:35.673346    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/35e3fbe6-ec61-4186-bfcb-481d52cfde31-home" (OuterVolumeSpecName: "home") pod "35e3fbe6-ec61-4186-bfcb-481d52cfde31" (UID: "35e3fbe6-ec61-4186-bfcb-481d52cfde31"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 23:04:35.673486 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:04:35.673460    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/35e3fbe6-ec61-4186-bfcb-481d52cfde31-model-cache" (OuterVolumeSpecName: "model-cache") pod "35e3fbe6-ec61-4186-bfcb-481d52cfde31" (UID: "35e3fbe6-ec61-4186-bfcb-481d52cfde31"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 23:04:35.675653 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:04:35.675604    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/35e3fbe6-ec61-4186-bfcb-481d52cfde31-dshm" (OuterVolumeSpecName: "dshm") pod "35e3fbe6-ec61-4186-bfcb-481d52cfde31" (UID: "35e3fbe6-ec61-4186-bfcb-481d52cfde31"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 23:04:35.675756 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:04:35.675699    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/35e3fbe6-ec61-4186-bfcb-481d52cfde31-kube-api-access-764d8" (OuterVolumeSpecName: "kube-api-access-764d8") pod "35e3fbe6-ec61-4186-bfcb-481d52cfde31" (UID: "35e3fbe6-ec61-4186-bfcb-481d52cfde31"). InnerVolumeSpecName "kube-api-access-764d8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 23:04:35.675845 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:04:35.675827    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/35e3fbe6-ec61-4186-bfcb-481d52cfde31-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "35e3fbe6-ec61-4186-bfcb-481d52cfde31" (UID: "35e3fbe6-ec61-4186-bfcb-481d52cfde31"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 23:04:35.773862 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:04:35.773827    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/35e3fbe6-ec61-4186-bfcb-481d52cfde31-home\") on node \"ip-10-0-133-105.ec2.internal\" DevicePath \"\""
Jul 06 23:04:35.773862 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:04:35.773856    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/35e3fbe6-ec61-4186-bfcb-481d52cfde31-tls-certs\") on node \"ip-10-0-133-105.ec2.internal\" DevicePath \"\""
Jul 06 23:04:35.774064 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:04:35.773872    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/35e3fbe6-ec61-4186-bfcb-481d52cfde31-model-cache\") on node \"ip-10-0-133-105.ec2.internal\" DevicePath \"\""
Jul 06 23:04:35.774064 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:04:35.773884    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/35e3fbe6-ec61-4186-bfcb-481d52cfde31-dshm\") on node \"ip-10-0-133-105.ec2.internal\" DevicePath \"\""
Jul 06 23:04:35.774064 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:04:35.773894    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-764d8\" (UniqueName: \"kubernetes.io/projected/35e3fbe6-ec61-4186-bfcb-481d52cfde31-kube-api-access-764d8\") on node \"ip-10-0-133-105.ec2.internal\" DevicePath \"\""
Jul 06 23:04:36.329254 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:04:36.329219    2563 generic.go:356] "Generic (PLEG): container finished" podID="35e3fbe6-ec61-4186-bfcb-481d52cfde31" containerID="bc1fb82c0bc2f56c690bc6123ac3ef1959e572b25d4b4303cc073adc7cdc3674" exitCode=0
Jul 06 23:04:36.329689 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:04:36.329268    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-b0de62/rate-test-model-b0de62-kserve-7969fb8bcf-gjkxl"
Jul 06 23:04:36.329689 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:04:36.329278    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-b0de62/rate-test-model-b0de62-kserve-7969fb8bcf-gjkxl" event={"ID":"35e3fbe6-ec61-4186-bfcb-481d52cfde31","Type":"ContainerDied","Data":"bc1fb82c0bc2f56c690bc6123ac3ef1959e572b25d4b4303cc073adc7cdc3674"}
Jul 06 23:04:36.329689 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:04:36.329310    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-b0de62/rate-test-model-b0de62-kserve-7969fb8bcf-gjkxl" event={"ID":"35e3fbe6-ec61-4186-bfcb-481d52cfde31","Type":"ContainerDied","Data":"56196d7e3ac65f20d3ee3ffee5fb5a5e87a38f9ff6cabe17726aaa98d4b387b5"}
Jul 06 23:04:36.329689 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:04:36.329326    2563 scope.go:117] "RemoveContainer" containerID="bc1fb82c0bc2f56c690bc6123ac3ef1959e572b25d4b4303cc073adc7cdc3674"
Jul 06 23:04:36.338254 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:04:36.338234    2563 scope.go:117] "RemoveContainer" containerID="bc1fb82c0bc2f56c690bc6123ac3ef1959e572b25d4b4303cc073adc7cdc3674"
Jul 06 23:04:36.338500 ip-10-0-133-105 kubenswrapper[2563]: E0706 23:04:36.338482    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bc1fb82c0bc2f56c690bc6123ac3ef1959e572b25d4b4303cc073adc7cdc3674\": container with ID starting with bc1fb82c0bc2f56c690bc6123ac3ef1959e572b25d4b4303cc073adc7cdc3674 not found: ID does not exist" containerID="bc1fb82c0bc2f56c690bc6123ac3ef1959e572b25d4b4303cc073adc7cdc3674"
Jul 06 23:04:36.338543 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:04:36.338510    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc1fb82c0bc2f56c690bc6123ac3ef1959e572b25d4b4303cc073adc7cdc3674"} err="failed to get container status \"bc1fb82c0bc2f56c690bc6123ac3ef1959e572b25d4b4303cc073adc7cdc3674\": rpc error: code = NotFound desc = could not find container \"bc1fb82c0bc2f56c690bc6123ac3ef1959e572b25d4b4303cc073adc7cdc3674\": container with ID starting with bc1fb82c0bc2f56c690bc6123ac3ef1959e572b25d4b4303cc073adc7cdc3674 not found: ID does not exist"
Jul 06 23:04:36.350306 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:04:36.350283    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-b0de62/rate-test-model-b0de62-kserve-7969fb8bcf-gjkxl"]
Jul 06 23:04:36.355861 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:04:36.355830    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-b0de62/rate-test-model-b0de62-kserve-7969fb8bcf-gjkxl"]
Jul 06 23:04:37.079801 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:04:37.079765    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="35e3fbe6-ec61-4186-bfcb-481d52cfde31" path="/var/lib/kubelet/pods/35e3fbe6-ec61-4186-bfcb-481d52cfde31/volumes"
Jul 06 23:05:04.983931 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:05:04.983894    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-b0de62-6df4b6c89-c5h58"]
Jul 06 23:05:04.984390 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:05:04.984205    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-shared-b-b0de62-6df4b6c89-c5h58" podUID="5a3baaa4-1dee-40fb-93ae-a6e93dccd2c5" containerName="maas-api" containerID="cri-o://175900dd1bf5f04541fb0bac3bb59a549bc54d386c53cc5c51720547a475eda5" gracePeriod=30
Jul 06 23:05:05.240161 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:05:05.240097    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-b0de62-6df4b6c89-c5h58"
Jul 06 23:05:05.325532 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:05:05.325499    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5a3baaa4-1dee-40fb-93ae-a6e93dccd2c5-maas-api-tls\") pod \"5a3baaa4-1dee-40fb-93ae-a6e93dccd2c5\" (UID: \"5a3baaa4-1dee-40fb-93ae-a6e93dccd2c5\") "
Jul 06 23:05:05.325711 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:05:05.325609    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-98n8n\" (UniqueName: \"kubernetes.io/projected/5a3baaa4-1dee-40fb-93ae-a6e93dccd2c5-kube-api-access-98n8n\") pod \"5a3baaa4-1dee-40fb-93ae-a6e93dccd2c5\" (UID: \"5a3baaa4-1dee-40fb-93ae-a6e93dccd2c5\") "
Jul 06 23:05:05.325711 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:05:05.325648    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5a3baaa4-1dee-40fb-93ae-a6e93dccd2c5-openshift-service-ca\") pod \"5a3baaa4-1dee-40fb-93ae-a6e93dccd2c5\" (UID: \"5a3baaa4-1dee-40fb-93ae-a6e93dccd2c5\") "
Jul 06 23:05:05.326040 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:05:05.326015    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5a3baaa4-1dee-40fb-93ae-a6e93dccd2c5-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "5a3baaa4-1dee-40fb-93ae-a6e93dccd2c5" (UID: "5a3baaa4-1dee-40fb-93ae-a6e93dccd2c5"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 23:05:05.327710 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:05:05.327690    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5a3baaa4-1dee-40fb-93ae-a6e93dccd2c5-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "5a3baaa4-1dee-40fb-93ae-a6e93dccd2c5" (UID: "5a3baaa4-1dee-40fb-93ae-a6e93dccd2c5"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 23:05:05.327776 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:05:05.327744    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5a3baaa4-1dee-40fb-93ae-a6e93dccd2c5-kube-api-access-98n8n" (OuterVolumeSpecName: "kube-api-access-98n8n") pod "5a3baaa4-1dee-40fb-93ae-a6e93dccd2c5" (UID: "5a3baaa4-1dee-40fb-93ae-a6e93dccd2c5"). InnerVolumeSpecName "kube-api-access-98n8n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 23:05:05.426719 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:05:05.426682    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5a3baaa4-1dee-40fb-93ae-a6e93dccd2c5-maas-api-tls\") on node \"ip-10-0-133-105.ec2.internal\" DevicePath \"\""
Jul 06 23:05:05.426719 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:05:05.426714    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-98n8n\" (UniqueName: \"kubernetes.io/projected/5a3baaa4-1dee-40fb-93ae-a6e93dccd2c5-kube-api-access-98n8n\") on node \"ip-10-0-133-105.ec2.internal\" DevicePath \"\""
Jul 06 23:05:05.426719 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:05:05.426726    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5a3baaa4-1dee-40fb-93ae-a6e93dccd2c5-openshift-service-ca\") on node \"ip-10-0-133-105.ec2.internal\" DevicePath \"\""
Jul 06 23:05:05.469827 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:05:05.469792    2563 generic.go:356] "Generic (PLEG): container finished" podID="5a3baaa4-1dee-40fb-93ae-a6e93dccd2c5" containerID="175900dd1bf5f04541fb0bac3bb59a549bc54d386c53cc5c51720547a475eda5" exitCode=0
Jul 06 23:05:05.469977 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:05:05.469842    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-b0de62-6df4b6c89-c5h58"
Jul 06 23:05:05.469977 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:05:05.469867    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-b0de62-6df4b6c89-c5h58" event={"ID":"5a3baaa4-1dee-40fb-93ae-a6e93dccd2c5","Type":"ContainerDied","Data":"175900dd1bf5f04541fb0bac3bb59a549bc54d386c53cc5c51720547a475eda5"}
Jul 06 23:05:05.469977 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:05:05.469917    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-b0de62-6df4b6c89-c5h58" event={"ID":"5a3baaa4-1dee-40fb-93ae-a6e93dccd2c5","Type":"ContainerDied","Data":"ce4562c56efdc96f1309f5d0fb90f22562833d5046abf2b06ebe202eb34a0f30"}
Jul 06 23:05:05.469977 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:05:05.469949    2563 scope.go:117] "RemoveContainer" containerID="175900dd1bf5f04541fb0bac3bb59a549bc54d386c53cc5c51720547a475eda5"
Jul 06 23:05:05.481217 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:05:05.481195    2563 scope.go:117] "RemoveContainer" containerID="175900dd1bf5f04541fb0bac3bb59a549bc54d386c53cc5c51720547a475eda5"
Jul 06 23:05:05.481484 ip-10-0-133-105 kubenswrapper[2563]: E0706 23:05:05.481460    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"175900dd1bf5f04541fb0bac3bb59a549bc54d386c53cc5c51720547a475eda5\": container with ID starting with 175900dd1bf5f04541fb0bac3bb59a549bc54d386c53cc5c51720547a475eda5 not found: ID does not exist" containerID="175900dd1bf5f04541fb0bac3bb59a549bc54d386c53cc5c51720547a475eda5"
Jul 06 23:05:05.481602 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:05:05.481496    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"175900dd1bf5f04541fb0bac3bb59a549bc54d386c53cc5c51720547a475eda5"} err="failed to get container status \"175900dd1bf5f04541fb0bac3bb59a549bc54d386c53cc5c51720547a475eda5\": rpc error: code = NotFound desc = could not find container \"175900dd1bf5f04541fb0bac3bb59a549bc54d386c53cc5c51720547a475eda5\": container with ID starting with 175900dd1bf5f04541fb0bac3bb59a549bc54d386c53cc5c51720547a475eda5 not found: ID does not exist"
Jul 06 23:05:05.494325 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:05:05.494260    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-b0de62-6df4b6c89-c5h58"]
Jul 06 23:05:05.497487 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:05:05.497463    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-b0de62-6df4b6c89-c5h58"]
Jul 06 23:05:07.080644 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:05:07.080609    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5a3baaa4-1dee-40fb-93ae-a6e93dccd2c5" path="/var/lib/kubelet/pods/5a3baaa4-1dee-40fb-93ae-a6e93dccd2c5/volumes"
Jul 06 23:05:07.475213 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:05:07.475160    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-9q89m"]
Jul 06 23:05:17.131161 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:05:17.131134    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-chgfg_42e3efe9-17e9-4b0f-8ef6-f3b7922ae8a5/manager/0.log"
Jul 06 23:05:17.250461 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:05:17.250420    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-mmb27_b7051b49-7c43-4cab-a533-2e2c9b64fddf/manager/0.log"
Jul 06 23:05:17.488360 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:05:17.488331    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29722965-ht74g_f0e88425-ee44-47d8-bac2-df42a5582f64/cleanup/0.log"
Jul 06 23:05:17.712804 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:05:17.712776    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6dccc7f4ff-4vrzw_9d2b9b0b-ce4e-431d-b762-87dc80ad88a0/manager/0.log"
Jul 06 23:05:17.826871 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:05:17.826800    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-r5zbk_ffd77de9-663b-495f-9b8d-d744c8145a03/server/0.log"
Jul 06 23:05:18.053279 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:05:18.053253    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-85fcf54485-9lvpp_e4aeb198-110f-4a6e-99dc-d3df527d8fc0/copy-manifests/0.log"
Jul 06 23:05:18.070846 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:05:18.067863    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-85fcf54485-9lvpp_e4aeb198-110f-4a6e-99dc-d3df527d8fc0/manager/0.log"
Jul 06 23:05:19.810447 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:05:19.810419    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-bwrkv_3bcdb607-d473-4695-b8d2-fc3431446f95/manager/0.log"
Jul 06 23:05:20.275021 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:05:20.274992    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-fcqxm_dff0d5e7-693e-498e-8ffe-fadbfff8635f/manager/0.log"
Jul 06 23:05:20.399402 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:05:20.399374    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-9q89m_53a67b97-ec4a-4505-8fd7-fc6bec48335a/limitador/0.log"
Jul 06 23:05:20.988494 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:05:20.988464    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-z2tm5_8fe2c7e3-c0da-4480-90f8-d17046019fdd/discovery/0.log"
Jul 06 23:05:21.092347 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:05:21.092316    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7f6975f4cc-hmkl4_bfd0e5f4-30d1-4e01-acb1-e6f254227584/kube-auth-proxy/0.log"
Jul 06 23:05:22.491458 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:05:22.491426    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-52s95_97bcd776-42fd-449d-8194-48acdbbdfb28/main/0.log"
Jul 06 23:05:29.655099 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:05:29.655070    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-mc48r_aad76575-952c-4b85-b64a-2523529c1cf4/global-pull-secret-syncer/0.log"
Jul 06 23:05:29.815927 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:05:29.815897    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-z88v4_686a5812-1550-4164-8e13-080cf1953bdb/konnectivity-agent/0.log"
Jul 06 23:05:29.836732 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:05:29.836703    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-133-105.ec2.internal_80e55e8f9c22a5bcba25f5b76deba1dd/haproxy/0.log"
Jul 06 23:05:35.040106 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:05:35.040076    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-bwrkv_3bcdb607-d473-4695-b8d2-fc3431446f95/manager/0.log"
Jul 06 23:05:35.275878 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:05:35.275793    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-fcqxm_dff0d5e7-693e-498e-8ffe-fadbfff8635f/manager/0.log"
Jul 06 23:05:35.303061 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:05:35.303032    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-9q89m_53a67b97-ec4a-4505-8fd7-fc6bec48335a/limitador/0.log"
Jul 06 23:05:37.291142 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:05:37.291043    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-zldgb_f098e977-5c44-4797-b931-19ab8d0165a7/kube-state-metrics/0.log"
Jul 06 23:05:37.311481 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:05:37.311456    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-zldgb_f098e977-5c44-4797-b931-19ab8d0165a7/kube-rbac-proxy-main/0.log"
Jul 06 23:05:37.333436 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:05:37.333416    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-zldgb_f098e977-5c44-4797-b931-19ab8d0165a7/kube-rbac-proxy-self/0.log"
Jul 06 23:05:37.389000 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:05:37.388971    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-d5dcff7cf-p4fvh_a0f70680-ae20-4ec1-94dd-a29c958a8d53/monitoring-plugin/0.log"
Jul 06 23:05:37.578471 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:05:37.578401    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-gzfm7_27c8bc73-d90a-4cee-b0f4-28c7b6d06012/node-exporter/0.log"
Jul 06 23:05:37.601959 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:05:37.601936    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-gzfm7_27c8bc73-d90a-4cee-b0f4-28c7b6d06012/kube-rbac-proxy/0.log"
Jul 06 23:05:37.624891 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:05:37.624866    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-gzfm7_27c8bc73-d90a-4cee-b0f4-28c7b6d06012/init-textfile/0.log"
Jul 06 23:05:37.740345 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:05:37.740318    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_50dfcfd6-7883-4a00-ae83-75eda98d73f8/prometheus/0.log"
Jul 06 23:05:37.774854 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:05:37.774824    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_50dfcfd6-7883-4a00-ae83-75eda98d73f8/config-reloader/0.log"
Jul 06 23:05:37.798127 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:05:37.798106    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_50dfcfd6-7883-4a00-ae83-75eda98d73f8/thanos-sidecar/0.log"
Jul 06 23:05:37.819770 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:05:37.819747    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_50dfcfd6-7883-4a00-ae83-75eda98d73f8/kube-rbac-proxy-web/0.log"
Jul 06 23:05:37.840934 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:05:37.840854    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_50dfcfd6-7883-4a00-ae83-75eda98d73f8/kube-rbac-proxy/0.log"
Jul 06 23:05:37.861918 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:05:37.861896    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_50dfcfd6-7883-4a00-ae83-75eda98d73f8/kube-rbac-proxy-thanos/0.log"
Jul 06 23:05:37.882161 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:05:37.882139    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_50dfcfd6-7883-4a00-ae83-75eda98d73f8/init-config-reloader/0.log"
Jul 06 23:05:37.910581 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:05:37.910561    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-hkrrp_b95fccd2-4822-4419-b9ba-51764b5edfc6/prometheus-operator/0.log"
Jul 06 23:05:37.929714 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:05:37.929699    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-hkrrp_b95fccd2-4822-4419-b9ba-51764b5edfc6/kube-rbac-proxy/0.log"
Jul 06 23:05:37.953319 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:05:37.953298    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-d559d66d5-btxqr_080c7027-a067-4d22-9bff-c818bca84503/prometheus-operator-admission-webhook/0.log"
Jul 06 23:05:39.953977 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:05:39.953939    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-8lsnx/perf-node-gather-daemonset-v4nnw"]
Jul 06 23:05:39.954412 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:05:39.954289    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5a3baaa4-1dee-40fb-93ae-a6e93dccd2c5" containerName="maas-api"
Jul 06 23:05:39.954412 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:05:39.954301    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="5a3baaa4-1dee-40fb-93ae-a6e93dccd2c5" containerName="maas-api"
Jul 06 23:05:39.954412 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:05:39.954310    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="671fae4f-a3e8-442d-b0c8-15b3e69d290a" containerName="maas-api"
Jul 06 23:05:39.954412 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:05:39.954318    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="671fae4f-a3e8-442d-b0c8-15b3e69d290a" containerName="maas-api"
Jul 06 23:05:39.954412 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:05:39.954324    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="35e3fbe6-ec61-4186-bfcb-481d52cfde31" containerName="main"
Jul 06 23:05:39.954412 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:05:39.954329    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="35e3fbe6-ec61-4186-bfcb-481d52cfde31" containerName="main"
Jul 06 23:05:39.954412 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:05:39.954390    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="5a3baaa4-1dee-40fb-93ae-a6e93dccd2c5" containerName="maas-api"
Jul 06 23:05:39.954412 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:05:39.954401    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="35e3fbe6-ec61-4186-bfcb-481d52cfde31" containerName="main"
Jul 06 23:05:39.954412 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:05:39.954407    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="671fae4f-a3e8-442d-b0c8-15b3e69d290a" containerName="maas-api"
Jul 06 23:05:39.957647 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:05:39.957626    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-8lsnx/perf-node-gather-daemonset-v4nnw"
Jul 06 23:05:39.960222 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:05:39.960200    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-8lsnx\"/\"openshift-service-ca.crt\""
Jul 06 23:05:39.961213 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:05:39.961197    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-8lsnx\"/\"default-dockercfg-h582k\""
Jul 06 23:05:39.961309 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:05:39.961217    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-8lsnx\"/\"kube-root-ca.crt\""
Jul 06 23:05:39.965149 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:05:39.965131    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-8lsnx/perf-node-gather-daemonset-v4nnw"]
Jul 06 23:05:40.032124 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:05:40.032089    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6642a897-b8de-400c-9098-539d31168560-lib-modules\") pod \"perf-node-gather-daemonset-v4nnw\" (UID: \"6642a897-b8de-400c-9098-539d31168560\") " pod="openshift-must-gather-8lsnx/perf-node-gather-daemonset-v4nnw"
Jul 06 23:05:40.032324 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:05:40.032137    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/6642a897-b8de-400c-9098-539d31168560-podres\") pod \"perf-node-gather-daemonset-v4nnw\" (UID: \"6642a897-b8de-400c-9098-539d31168560\") " pod="openshift-must-gather-8lsnx/perf-node-gather-daemonset-v4nnw"
Jul 06 23:05:40.032324 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:05:40.032192    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dp8rv\" (UniqueName: \"kubernetes.io/projected/6642a897-b8de-400c-9098-539d31168560-kube-api-access-dp8rv\") pod \"perf-node-gather-daemonset-v4nnw\" (UID: \"6642a897-b8de-400c-9098-539d31168560\") " pod="openshift-must-gather-8lsnx/perf-node-gather-daemonset-v4nnw"
Jul 06 23:05:40.032324 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:05:40.032234    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/6642a897-b8de-400c-9098-539d31168560-proc\") pod \"perf-node-gather-daemonset-v4nnw\" (UID: \"6642a897-b8de-400c-9098-539d31168560\") " pod="openshift-must-gather-8lsnx/perf-node-gather-daemonset-v4nnw"
Jul 06 23:05:40.032324 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:05:40.032252    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6642a897-b8de-400c-9098-539d31168560-sys\") pod \"perf-node-gather-daemonset-v4nnw\" (UID: \"6642a897-b8de-400c-9098-539d31168560\") " pod="openshift-must-gather-8lsnx/perf-node-gather-daemonset-v4nnw"
Jul 06 23:05:40.132938 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:05:40.132901    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6642a897-b8de-400c-9098-539d31168560-lib-modules\") pod \"perf-node-gather-daemonset-v4nnw\" (UID: \"6642a897-b8de-400c-9098-539d31168560\") " pod="openshift-must-gather-8lsnx/perf-node-gather-daemonset-v4nnw"
Jul 06 23:05:40.133092 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:05:40.132946    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/6642a897-b8de-400c-9098-539d31168560-podres\") pod \"perf-node-gather-daemonset-v4nnw\" (UID: \"6642a897-b8de-400c-9098-539d31168560\") " pod="openshift-must-gather-8lsnx/perf-node-gather-daemonset-v4nnw"
Jul 06 23:05:40.133092 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:05:40.132971    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dp8rv\" (UniqueName: \"kubernetes.io/projected/6642a897-b8de-400c-9098-539d31168560-kube-api-access-dp8rv\") pod \"perf-node-gather-daemonset-v4nnw\" (UID: \"6642a897-b8de-400c-9098-539d31168560\") " pod="openshift-must-gather-8lsnx/perf-node-gather-daemonset-v4nnw"
Jul 06 23:05:40.133092 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:05:40.133017    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/6642a897-b8de-400c-9098-539d31168560-proc\") pod \"perf-node-gather-daemonset-v4nnw\" (UID: \"6642a897-b8de-400c-9098-539d31168560\") " pod="openshift-must-gather-8lsnx/perf-node-gather-daemonset-v4nnw"
Jul 06 23:05:40.133092 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:05:40.133044    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6642a897-b8de-400c-9098-539d31168560-sys\") pod \"perf-node-gather-daemonset-v4nnw\" (UID: \"6642a897-b8de-400c-9098-539d31168560\") " pod="openshift-must-gather-8lsnx/perf-node-gather-daemonset-v4nnw"
Jul 06 23:05:40.133092 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:05:40.133078    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/6642a897-b8de-400c-9098-539d31168560-podres\") pod \"perf-node-gather-daemonset-v4nnw\" (UID: \"6642a897-b8de-400c-9098-539d31168560\") " pod="openshift-must-gather-8lsnx/perf-node-gather-daemonset-v4nnw"
Jul 06 23:05:40.133369 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:05:40.133078    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6642a897-b8de-400c-9098-539d31168560-lib-modules\") pod \"perf-node-gather-daemonset-v4nnw\" (UID: \"6642a897-b8de-400c-9098-539d31168560\") " pod="openshift-must-gather-8lsnx/perf-node-gather-daemonset-v4nnw"
Jul 06 23:05:40.133369 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:05:40.133115    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/6642a897-b8de-400c-9098-539d31168560-proc\") pod \"perf-node-gather-daemonset-v4nnw\" (UID: \"6642a897-b8de-400c-9098-539d31168560\") " pod="openshift-must-gather-8lsnx/perf-node-gather-daemonset-v4nnw"
Jul 06 23:05:40.133369 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:05:40.133163    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6642a897-b8de-400c-9098-539d31168560-sys\") pod \"perf-node-gather-daemonset-v4nnw\" (UID: \"6642a897-b8de-400c-9098-539d31168560\") " pod="openshift-must-gather-8lsnx/perf-node-gather-daemonset-v4nnw"
Jul 06 23:05:40.140612 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:05:40.140592    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dp8rv\" (UniqueName: \"kubernetes.io/projected/6642a897-b8de-400c-9098-539d31168560-kube-api-access-dp8rv\") pod \"perf-node-gather-daemonset-v4nnw\" (UID: \"6642a897-b8de-400c-9098-539d31168560\") " pod="openshift-must-gather-8lsnx/perf-node-gather-daemonset-v4nnw"
Jul 06 23:05:40.268989 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:05:40.268913    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-8lsnx/perf-node-gather-daemonset-v4nnw"
Jul 06 23:05:40.397680 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:05:40.397610    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-8lsnx/perf-node-gather-daemonset-v4nnw"]
Jul 06 23:05:40.401746 ip-10-0-133-105 kubenswrapper[2563]: W0706 23:05:40.401720    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod6642a897_b8de_400c_9098_539d31168560.slice/crio-d1cc136bfd2a64482dfc908018454b6147553c04e6ee80b87ce71765a0d80d0e WatchSource:0}: Error finding container d1cc136bfd2a64482dfc908018454b6147553c04e6ee80b87ce71765a0d80d0e: Status 404 returned error can't find the container with id d1cc136bfd2a64482dfc908018454b6147553c04e6ee80b87ce71765a0d80d0e
Jul 06 23:05:40.598782 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:05:40.598751    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-685bfb5f64-lgzh4_11551149-01c0-4656-be37-d24965aac4a4/download-server/0.log"
Jul 06 23:05:40.635265 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:05:40.635234    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8lsnx/perf-node-gather-daemonset-v4nnw" event={"ID":"6642a897-b8de-400c-9098-539d31168560","Type":"ContainerStarted","Data":"c825e453c51b1fc0d860318571e63acd70651887e8f05638b8fb4717daede625"}
Jul 06 23:05:40.635426 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:05:40.635286    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8lsnx/perf-node-gather-daemonset-v4nnw" event={"ID":"6642a897-b8de-400c-9098-539d31168560","Type":"ContainerStarted","Data":"d1cc136bfd2a64482dfc908018454b6147553c04e6ee80b87ce71765a0d80d0e"}
Jul 06 23:05:40.635426 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:05:40.635309    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-8lsnx/perf-node-gather-daemonset-v4nnw"
Jul 06 23:05:40.652588 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:05:40.652511    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-8lsnx/perf-node-gather-daemonset-v4nnw" podStartSLOduration=1.652484989 podStartE2EDuration="1.652484989s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 23:05:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:05:40.650536168 +0000 UTC m=+2506.075582701" watchObservedRunningTime="2026-07-06 23:05:40.652484989 +0000 UTC m=+2506.077531521"
Jul 06 23:05:42.217850 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:05:42.217824    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-bv5xq_62bbe928-f2d0-4642-a2a3-f272341820eb/dns/0.log"
Jul 06 23:05:42.241375 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:05:42.241349    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-bv5xq_62bbe928-f2d0-4642-a2a3-f272341820eb/kube-rbac-proxy/0.log"
Jul 06 23:05:42.343916 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:05:42.343892    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-tgnb8_9c27bdd9-eca8-4ef2-a260-56bb21de6b1e/dns-node-resolver/0.log"
Jul 06 23:05:42.888362 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:05:42.888331    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-745f45d84c-z5h56_27e84944-f883-4de2-bd21-2577c8e8d01b/registry/0.log"
Jul 06 23:05:42.928624 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:05:42.928597    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-9l9l5_6a3050e9-b393-4767-a9c2-dafdf2a17925/node-ca/0.log"
Jul 06 23:05:44.023120 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:05:44.023085    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-z2tm5_8fe2c7e3-c0da-4480-90f8-d17046019fdd/discovery/0.log"
Jul 06 23:05:44.045135 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:05:44.045106    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7f6975f4cc-hmkl4_bfd0e5f4-30d1-4e01-acb1-e6f254227584/kube-auth-proxy/0.log"
Jul 06 23:05:44.933215 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:05:44.933187    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-78psz_6e143322-bd8b-42e9-ab2b-7071044779a8/serve-healthcheck-canary/0.log"
Jul 06 23:05:45.538232 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:05:45.538204    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-jxjpn_5e411fd6-9619-4fb9-a556-f20259e121e0/kube-rbac-proxy/0.log"
Jul 06 23:05:45.564478 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:05:45.564437    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-jxjpn_5e411fd6-9619-4fb9-a556-f20259e121e0/exporter/0.log"
Jul 06 23:05:45.586651 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:05:45.586623    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-jxjpn_5e411fd6-9619-4fb9-a556-f20259e121e0/extractor/0.log"
Jul 06 23:05:46.649992 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:05:46.649913    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-8lsnx/perf-node-gather-daemonset-v4nnw"
Jul 06 23:05:47.732790 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:05:47.732765    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-chgfg_42e3efe9-17e9-4b0f-8ef6-f3b7922ae8a5/manager/0.log"
Jul 06 23:05:47.864044 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:05:47.864011    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-mmb27_b7051b49-7c43-4cab-a533-2e2c9b64fddf/manager/0.log"
Jul 06 23:05:47.926115 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:05:47.926081    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29722965-ht74g_f0e88425-ee44-47d8-bac2-df42a5582f64/cleanup/0.log"
Jul 06 23:05:48.043835 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:05:48.043756    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6dccc7f4ff-4vrzw_9d2b9b0b-ce4e-431d-b762-87dc80ad88a0/manager/0.log"
Jul 06 23:05:48.064220 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:05:48.064194    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-r5zbk_ffd77de9-663b-495f-9b8d-d744c8145a03/server/0.log"
Jul 06 23:05:48.200454 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:05:48.200423    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-85fcf54485-9lvpp_e4aeb198-110f-4a6e-99dc-d3df527d8fc0/manager/0.log"
Jul 06 23:05:48.216518 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:05:48.216489    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-85fcf54485-9lvpp_e4aeb198-110f-4a6e-99dc-d3df527d8fc0/copy-manifests/0.log"
Jul 06 23:05:50.347963 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:05:50.347884    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-z2tm5_8fe2c7e3-c0da-4480-90f8-d17046019fdd/discovery/0.log"
Jul 06 23:05:50.358894 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:05:50.358870    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7f6975f4cc-hmkl4_bfd0e5f4-30d1-4e01-acb1-e6f254227584/kube-auth-proxy/0.log"
Jul 06 23:05:51.784858 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:05:51.784829    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-z2tm5_8fe2c7e3-c0da-4480-90f8-d17046019fdd/discovery/0.log"
Jul 06 23:05:51.797578 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:05:51.797554    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7f6975f4cc-hmkl4_bfd0e5f4-30d1-4e01-acb1-e6f254227584/kube-auth-proxy/0.log"
Jul 06 23:05:56.673623 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:05:56.673586    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rzszm_9902a376-c450-4867-9c7c-d9e5300bce11/kube-multus-additional-cni-plugins/0.log"
Jul 06 23:05:56.693741 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:05:56.693720    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rzszm_9902a376-c450-4867-9c7c-d9e5300bce11/egress-router-binary-copy/0.log"
Jul 06 23:05:56.714427 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:05:56.714399    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rzszm_9902a376-c450-4867-9c7c-d9e5300bce11/cni-plugins/0.log"
Jul 06 23:05:56.735483 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:05:56.735457    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rzszm_9902a376-c450-4867-9c7c-d9e5300bce11/bond-cni-plugin/0.log"
Jul 06 23:05:56.755078 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:05:56.755057    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rzszm_9902a376-c450-4867-9c7c-d9e5300bce11/routeoverride-cni/0.log"
Jul 06 23:05:56.775893 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:05:56.775869    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rzszm_9902a376-c450-4867-9c7c-d9e5300bce11/whereabouts-cni-bincopy/0.log"
Jul 06 23:05:56.799094 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:05:56.799073    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rzszm_9902a376-c450-4867-9c7c-d9e5300bce11/whereabouts-cni/0.log"
Jul 06 23:05:56.858622 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:05:56.858594    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-vs62k_79685e2c-e9e0-4ebe-93e6-12dd50c3c75c/kube-multus/0.log"
Jul 06 23:05:56.920385 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:05:56.920356    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-bbjgn_69b0b78e-e934-433b-bfdb-5fd6d8ba4a29/network-metrics-daemon/0.log"
Jul 06 23:05:56.938926 ip-10-0-133-105 kubenswrapper[2563]: I0706 23:05:56.938875    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-bbjgn_69b0b78e-e934-433b-bfdb-5fd6d8ba4a29/kube-rbac-proxy/0.log"