Jul 07 11:11:15.483290 ip-10-0-136-15 systemd[1]: Starting Kubernetes Kubelet... Jul 07 11:11:15.850646 ip-10-0-136-15 kubenswrapper[2562]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 07 11:11:15.850646 ip-10-0-136-15 kubenswrapper[2562]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 07 11:11:15.850646 ip-10-0-136-15 kubenswrapper[2562]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 07 11:11:15.850646 ip-10-0-136-15 kubenswrapper[2562]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 07 11:11:15.850646 ip-10-0-136-15 kubenswrapper[2562]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 07 11:11:15.850646 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.833142 2562 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 07 11:11:15.850646 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.838069 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 11:11:15.850646 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.838081 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 11:11:15.850646 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.838085 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 11:11:15.850646 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.838088 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 11:11:15.850646 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.838091 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 11:11:15.850646 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.838094 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 11:11:15.850646 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.838097 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 11:11:15.850646 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.838100 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 11:11:15.852789 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.838103 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 11:11:15.852789 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.838106 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 11:11:15.852789 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.838109 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 11:11:15.852789 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.838111 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 11:11:15.852789 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.838114 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 11:11:15.852789 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.838116 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 11:11:15.852789 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.838120 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 11:11:15.852789 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.838128 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 11:11:15.852789 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.838131 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 11:11:15.852789 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.838134 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 11:11:15.852789 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.838136 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 11:11:15.852789 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.838139 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 11:11:15.852789 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.838142 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 11:11:15.852789 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.838145 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 11:11:15.852789 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.838147 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 11:11:15.852789 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.838150 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 11:11:15.852789 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.838152 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 11:11:15.852789 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.838155 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 11:11:15.852789 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.838158 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 11:11:15.852789 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.838160 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 11:11:15.855259 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.838163 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 11:11:15.855259 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.838165 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 11:11:15.855259 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.838168 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 11:11:15.855259 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.838170 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 11:11:15.855259 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.838173 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 11:11:15.855259 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.838175 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 11:11:15.855259 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.838178 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 11:11:15.855259 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.838182 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 11:11:15.855259 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.838185 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 11:11:15.855259 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.838187 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 11:11:15.855259 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.838190 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 11:11:15.855259 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.838195 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 11:11:15.855259 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.838199 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 11:11:15.855259 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.838202 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 11:11:15.855259 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.838206 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 11:11:15.855259 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.838210 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 11:11:15.855259 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.838212 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 11:11:15.855259 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.838215 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 11:11:15.855259 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.838218 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 11:11:15.856504 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.838221 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 11:11:15.856504 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.838224 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 11:11:15.856504 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.838227 2562 feature_gate.go:328] unrecognized feature gate: Example Jul 07 11:11:15.856504 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.838231 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 11:11:15.856504 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.838234 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 11:11:15.856504 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.838237 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 11:11:15.856504 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.838239 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 11:11:15.856504 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.838242 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 11:11:15.856504 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.838245 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 11:11:15.856504 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.838247 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 11:11:15.856504 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.838250 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 11:11:15.856504 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.838252 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 11:11:15.856504 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.838255 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 11:11:15.856504 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.838257 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 11:11:15.856504 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.838259 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 11:11:15.856504 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.838262 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 11:11:15.856504 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.838264 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 11:11:15.856504 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.838267 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 11:11:15.856504 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.838270 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 11:11:15.856504 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.838273 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 11:11:15.857167 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.838275 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 11:11:15.857167 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.838278 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 11:11:15.857167 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.838281 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 11:11:15.857167 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.838283 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 11:11:15.857167 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.838288 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 11:11:15.857167 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.838291 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 11:11:15.857167 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.838295 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 11:11:15.857167 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.838297 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 11:11:15.857167 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.838300 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 11:11:15.857167 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.838303 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 11:11:15.857167 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.838305 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 11:11:15.857167 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.838308 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 11:11:15.857167 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.838310 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 11:11:15.857167 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.838313 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 11:11:15.857167 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.838316 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 11:11:15.857167 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.838319 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 11:11:15.857167 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.838321 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 11:11:15.857167 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.838324 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 11:11:15.857167 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.838326 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 11:11:15.858449 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.838329 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 11:11:15.858449 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.838331 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 11:11:15.858449 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.838334 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 11:11:15.858449 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.838337 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 11:11:15.858449 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.838339 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 11:11:15.858449 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.838342 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 11:11:15.858449 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.838344 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 11:11:15.858449 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.838347 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 11:11:15.858449 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.838350 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 11:11:15.858449 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.838352 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 11:11:15.858449 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.838355 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 11:11:15.858449 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.838357 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 11:11:15.858449 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.838360 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 11:11:15.858449 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839050 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 11:11:15.858449 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839057 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 11:11:15.858449 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839060 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 11:11:15.858449 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839062 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 11:11:15.858449 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839065 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 11:11:15.858449 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839069 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 11:11:15.858449 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839071 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 11:11:15.859065 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839074 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 11:11:15.859065 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839076 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 11:11:15.859065 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839079 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 11:11:15.859065 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839081 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 11:11:15.859065 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839084 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 11:11:15.859065 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839086 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 11:11:15.859065 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839089 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 11:11:15.859065 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839092 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 11:11:15.859065 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839094 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 11:11:15.859065 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839097 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 11:11:15.859065 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839099 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 11:11:15.859065 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839104 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 11:11:15.859065 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839107 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 11:11:15.859065 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839111 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 11:11:15.859065 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839114 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 11:11:15.859065 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839117 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 11:11:15.859065 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839120 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 11:11:15.859065 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839124 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 11:11:15.859065 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839127 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 11:11:15.859065 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839129 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 11:11:15.859594 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839132 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 11:11:15.859594 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839135 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 11:11:15.859594 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839138 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 11:11:15.859594 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839141 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 11:11:15.859594 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839144 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 11:11:15.859594 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839147 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 11:11:15.859594 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839149 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 11:11:15.859594 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839152 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 11:11:15.859594 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839154 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 11:11:15.859594 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839157 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 11:11:15.859594 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839159 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 11:11:15.859594 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839165 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 11:11:15.859594 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839168 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 11:11:15.859594 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839171 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 11:11:15.859594 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839173 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 11:11:15.859594 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839179 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 11:11:15.859594 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839181 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 11:11:15.859594 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839184 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 11:11:15.859594 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839187 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 11:11:15.859594 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839189 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 11:11:15.861834 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839192 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 11:11:15.861834 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839194 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 11:11:15.861834 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839197 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 11:11:15.861834 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839199 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 11:11:15.861834 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839203 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 11:11:15.861834 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839206 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 11:11:15.861834 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839209 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 11:11:15.861834 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839211 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 11:11:15.861834 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839214 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 11:11:15.861834 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839217 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 11:11:15.861834 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839220 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 11:11:15.861834 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839222 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 11:11:15.861834 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839225 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 11:11:15.861834 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839227 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 11:11:15.861834 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839230 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 11:11:15.861834 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839232 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 11:11:15.861834 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839235 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 11:11:15.861834 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839240 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 11:11:15.861834 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839245 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 11:11:15.862514 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839248 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 11:11:15.862514 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839251 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 11:11:15.862514 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839253 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 11:11:15.862514 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839256 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 11:11:15.862514 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839259 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 11:11:15.862514 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839262 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 11:11:15.862514 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839265 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 11:11:15.862514 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839267 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 11:11:15.862514 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839271 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 11:11:15.862514 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839274 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 11:11:15.862514 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839276 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 11:11:15.862514 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839278 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 11:11:15.862514 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839281 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 11:11:15.862514 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839284 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 11:11:15.862514 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839286 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 11:11:15.862514 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839288 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 11:11:15.862514 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839291 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 11:11:15.862514 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839293 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 11:11:15.862514 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839296 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 11:11:15.863762 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839298 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 11:11:15.863762 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839301 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 11:11:15.863762 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839303 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 11:11:15.863762 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839306 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 11:11:15.863762 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839308 2562 feature_gate.go:328] unrecognized feature gate: Example Jul 07 11:11:15.863762 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839311 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 11:11:15.863762 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839313 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 11:11:15.863762 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839316 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 11:11:15.863762 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839318 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 11:11:15.863762 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839321 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 11:11:15.863762 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839323 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 11:11:15.863762 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839325 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 11:11:15.863762 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839330 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 11:11:15.863762 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839332 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 11:11:15.863762 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839696 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 11:11:15.863762 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839701 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 11:11:15.863762 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839704 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 11:11:15.863762 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839706 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 11:11:15.863762 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839709 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 11:11:15.863762 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839712 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 11:11:15.866139 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839715 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 11:11:15.866139 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839719 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 11:11:15.866139 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839722 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 11:11:15.866139 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839724 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 11:11:15.866139 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839727 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 11:11:15.866139 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839729 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 11:11:15.866139 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839732 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 11:11:15.866139 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839734 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 11:11:15.866139 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839737 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 11:11:15.866139 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839739 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 11:11:15.866139 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839743 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 11:11:15.866139 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839747 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 11:11:15.866139 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839750 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 11:11:15.866139 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839752 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 11:11:15.866139 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839755 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 11:11:15.866139 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839758 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 11:11:15.866139 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839762 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 11:11:15.866139 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839765 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 11:11:15.866139 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839768 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 11:11:15.868416 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839772 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 11:11:15.868416 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839776 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 11:11:15.868416 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839779 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 11:11:15.868416 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839782 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 11:11:15.868416 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839785 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 11:11:15.868416 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839789 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 11:11:15.868416 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839792 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 11:11:15.868416 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839794 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 11:11:15.868416 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839798 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 11:11:15.868416 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839800 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 11:11:15.868416 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839802 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 11:11:15.868416 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839805 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 11:11:15.868416 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839809 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 11:11:15.868416 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839812 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 11:11:15.868416 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839815 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 11:11:15.868416 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839818 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 11:11:15.868416 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839820 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 11:11:15.868416 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839823 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 11:11:15.868416 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839825 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 11:11:15.868416 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839828 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 11:11:15.870634 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839830 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 11:11:15.870634 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839833 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 11:11:15.870634 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839835 2562 feature_gate.go:328] unrecognized feature gate: Example Jul 07 11:11:15.870634 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839838 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 11:11:15.870634 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839840 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 11:11:15.870634 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839844 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 11:11:15.870634 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839846 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 11:11:15.870634 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839849 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 11:11:15.870634 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839851 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 11:11:15.870634 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839853 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 11:11:15.870634 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839856 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 11:11:15.870634 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839858 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 11:11:15.870634 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839861 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 11:11:15.870634 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839863 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 11:11:15.870634 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839866 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 11:11:15.870634 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839868 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 11:11:15.870634 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839871 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 11:11:15.870634 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839876 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 11:11:15.870634 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839879 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 11:11:15.871409 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839881 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 11:11:15.871409 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839884 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 11:11:15.871409 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839886 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 11:11:15.871409 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839889 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 11:11:15.871409 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839891 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 11:11:15.871409 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839894 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 11:11:15.871409 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839896 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 11:11:15.871409 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839900 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 11:11:15.871409 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839903 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 11:11:15.871409 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839905 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 11:11:15.871409 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839908 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 11:11:15.871409 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839910 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 11:11:15.871409 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839932 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 11:11:15.871409 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839934 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 11:11:15.871409 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839937 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 11:11:15.871409 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839940 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 11:11:15.871409 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839942 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 11:11:15.871409 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839945 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 11:11:15.871409 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839947 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 11:11:15.871409 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839950 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 11:11:15.873529 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839953 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 11:11:15.873529 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839955 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 11:11:15.873529 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839958 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 11:11:15.873529 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839960 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 11:11:15.873529 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839963 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 11:11:15.873529 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839965 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 11:11:15.873529 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839968 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 11:11:15.873529 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839971 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 11:11:15.873529 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839973 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 11:11:15.873529 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839976 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 11:11:15.873529 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839980 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 11:11:15.873529 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839982 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 11:11:15.873529 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839985 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 11:11:15.873529 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839987 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 11:11:15.873529 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.839990 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 11:11:15.873529 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.840069 2562 flags.go:64] FLAG: --address="0.0.0.0" Jul 07 11:11:15.873529 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.840079 2562 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 07 11:11:15.873529 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.840087 2562 flags.go:64] FLAG: --anonymous-auth="true" Jul 07 11:11:15.873529 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.840094 2562 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 07 11:11:15.873529 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.840102 2562 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 07 11:11:15.873529 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.840107 2562 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 07 11:11:15.874969 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.840113 2562 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 07 11:11:15.874969 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.840117 2562 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 07 11:11:15.874969 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.840120 2562 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 07 11:11:15.874969 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.840123 2562 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 07 11:11:15.874969 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.840127 2562 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 07 11:11:15.874969 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.840130 2562 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 07 11:11:15.874969 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.840133 2562 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 07 11:11:15.874969 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.840136 2562 flags.go:64] FLAG: --cgroup-root="" Jul 07 11:11:15.874969 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.840139 2562 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 07 11:11:15.874969 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.840141 2562 flags.go:64] FLAG: --client-ca-file="" Jul 07 11:11:15.874969 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.840144 2562 flags.go:64] FLAG: --cloud-provider="external" Jul 07 11:11:15.874969 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.840147 2562 flags.go:64] FLAG: --cluster-dns="[]" Jul 07 11:11:15.874969 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.840152 2562 flags.go:64] FLAG: --cluster-domain="" Jul 07 11:11:15.874969 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.840155 2562 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 07 11:11:15.874969 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.840158 2562 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 07 11:11:15.874969 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.840161 2562 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 07 11:11:15.874969 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.840164 2562 flags.go:64] FLAG: --container-log-max-files="5" Jul 07 11:11:15.874969 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.840167 2562 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 07 11:11:15.874969 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.840170 2562 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 07 11:11:15.874969 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.840173 2562 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 07 11:11:15.874969 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.840176 2562 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 07 11:11:15.874969 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.840181 2562 flags.go:64] FLAG: --contention-profiling="false" Jul 07 11:11:15.874969 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.840184 2562 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 07 11:11:15.874969 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.840187 2562 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 07 11:11:15.876086 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.840190 2562 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 07 11:11:15.876086 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.840192 2562 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 07 11:11:15.876086 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.840196 2562 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 07 11:11:15.876086 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.840199 2562 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 07 11:11:15.876086 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.840202 2562 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 07 11:11:15.876086 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.840205 2562 flags.go:64] FLAG: --enable-load-reader="false" Jul 07 11:11:15.876086 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.840209 2562 flags.go:64] FLAG: --enable-server="true" Jul 07 11:11:15.876086 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.840212 2562 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 07 11:11:15.876086 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.840216 2562 flags.go:64] FLAG: --event-burst="100" Jul 07 11:11:15.876086 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.840219 2562 flags.go:64] FLAG: --event-qps="50" Jul 07 11:11:15.876086 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.840222 2562 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 07 11:11:15.876086 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.840225 2562 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 07 11:11:15.876086 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.840229 2562 flags.go:64] FLAG: --eviction-hard="" Jul 07 11:11:15.876086 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.840232 2562 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 07 11:11:15.876086 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.840235 2562 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 07 11:11:15.876086 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.840238 2562 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 07 11:11:15.876086 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.840241 2562 flags.go:64] FLAG: --eviction-soft="" Jul 07 11:11:15.876086 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.840244 2562 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 07 11:11:15.876086 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.840247 2562 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 07 11:11:15.876086 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.840249 2562 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 07 11:11:15.876086 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.840252 2562 flags.go:64] FLAG: --experimental-mounter-path="" Jul 07 11:11:15.876086 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.840255 2562 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 07 11:11:15.876086 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.840258 2562 flags.go:64] FLAG: --fail-swap-on="true" Jul 07 11:11:15.876086 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.840260 2562 flags.go:64] FLAG: --feature-gates="" Jul 07 11:11:15.876086 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.840264 2562 flags.go:64] FLAG: --file-check-frequency="20s" Jul 07 11:11:15.877343 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.840267 2562 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 07 11:11:15.877343 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.840270 2562 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 07 11:11:15.877343 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.840272 2562 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 07 11:11:15.877343 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.840275 2562 flags.go:64] FLAG: --healthz-port="10248" Jul 07 11:11:15.877343 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.840280 2562 flags.go:64] FLAG: --help="false" Jul 07 11:11:15.877343 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.840282 2562 flags.go:64] FLAG: --hostname-override="ip-10-0-136-15.ec2.internal" Jul 07 11:11:15.877343 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.840285 2562 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 07 11:11:15.877343 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.840288 2562 flags.go:64] FLAG: --http-check-frequency="20s" Jul 07 11:11:15.877343 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.840291 2562 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 07 11:11:15.877343 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.840294 2562 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 07 11:11:15.877343 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.840297 2562 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 07 11:11:15.877343 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.840300 2562 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 07 11:11:15.877343 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.840303 2562 flags.go:64] FLAG: --image-service-endpoint="" Jul 07 11:11:15.877343 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.840308 2562 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 07 11:11:15.877343 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.840310 2562 flags.go:64] FLAG: --kube-api-burst="100" Jul 07 11:11:15.877343 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.840314 2562 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 07 11:11:15.877343 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.840316 2562 flags.go:64] FLAG: --kube-api-qps="50" Jul 07 11:11:15.877343 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.840319 2562 flags.go:64] FLAG: --kube-reserved="" Jul 07 11:11:15.877343 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.840322 2562 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 07 11:11:15.877343 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.840325 2562 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 07 11:11:15.877343 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.840328 2562 flags.go:64] FLAG: --kubelet-cgroups="" Jul 07 11:11:15.877343 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.840331 2562 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 07 11:11:15.877343 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.840334 2562 flags.go:64] FLAG: --lock-file="" Jul 07 11:11:15.877343 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.840337 2562 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 07 11:11:15.878174 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.840339 2562 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 07 11:11:15.878174 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.840342 2562 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 07 11:11:15.878174 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.840347 2562 flags.go:64] FLAG: --log-json-split-stream="false" Jul 07 11:11:15.878174 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.840350 2562 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 07 11:11:15.878174 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.840353 2562 flags.go:64] FLAG: --log-text-split-stream="false" Jul 07 11:11:15.878174 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.840356 2562 flags.go:64] FLAG: --logging-format="text" Jul 07 11:11:15.878174 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.840358 2562 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 07 11:11:15.878174 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.840362 2562 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 07 11:11:15.878174 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.840364 2562 flags.go:64] FLAG: --manifest-url="" Jul 07 11:11:15.878174 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.840367 2562 flags.go:64] FLAG: --manifest-url-header="" Jul 07 11:11:15.878174 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.840371 2562 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 07 11:11:15.878174 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.840373 2562 flags.go:64] FLAG: --max-open-files="1000000" Jul 07 11:11:15.878174 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.840378 2562 flags.go:64] FLAG: --max-pods="110" Jul 07 11:11:15.878174 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.840381 2562 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 07 11:11:15.878174 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.840384 2562 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 07 11:11:15.878174 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.840387 2562 flags.go:64] FLAG: --memory-manager-policy="None" Jul 07 11:11:15.878174 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.840390 2562 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 07 11:11:15.878174 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.840393 2562 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 07 11:11:15.878174 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.840396 2562 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 07 11:11:15.878174 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.840398 2562 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 07 11:11:15.878174 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.840405 2562 flags.go:64] FLAG: --node-status-max-images="50" Jul 07 11:11:15.878174 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.840410 2562 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 07 11:11:15.878174 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.840412 2562 flags.go:64] FLAG: --oom-score-adj="-999" Jul 07 11:11:15.878174 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.840415 2562 flags.go:64] FLAG: --pod-cidr="" Jul 07 11:11:15.879225 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.840422 2562 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 07 11:11:15.879225 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.840427 2562 flags.go:64] FLAG: --pod-manifest-path="" Jul 07 11:11:15.879225 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.840430 2562 flags.go:64] FLAG: --pod-max-pids="-1" Jul 07 11:11:15.879225 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.840433 2562 flags.go:64] FLAG: --pods-per-core="0" Jul 07 11:11:15.879225 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.840436 2562 flags.go:64] FLAG: --port="10250" Jul 07 11:11:15.879225 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.840439 2562 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 07 11:11:15.879225 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.840441 2562 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0539d41618942ef72" Jul 07 11:11:15.879225 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.840444 2562 flags.go:64] FLAG: --qos-reserved="" Jul 07 11:11:15.879225 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.840447 2562 flags.go:64] FLAG: --read-only-port="10255" Jul 07 11:11:15.879225 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.840450 2562 flags.go:64] FLAG: --register-node="true" Jul 07 11:11:15.879225 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.840453 2562 flags.go:64] FLAG: --register-with-taints="" Jul 07 11:11:15.879225 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.840456 2562 flags.go:64] FLAG: --registry-burst="10" Jul 07 11:11:15.879225 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.840459 2562 flags.go:64] FLAG: --registry-qps="5" Jul 07 11:11:15.879225 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.840461 2562 flags.go:64] FLAG: --reserved-cpus="" Jul 07 11:11:15.879225 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.840464 2562 flags.go:64] FLAG: --reserved-memory="" Jul 07 11:11:15.879225 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.840468 2562 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 07 11:11:15.879225 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.840471 2562 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 07 11:11:15.879225 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.840474 2562 flags.go:64] FLAG: --rotate-certificates="false" Jul 07 11:11:15.879225 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.840477 2562 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 07 11:11:15.879225 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.840479 2562 flags.go:64] FLAG: --runonce="false" Jul 07 11:11:15.879225 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.840482 2562 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 07 11:11:15.879225 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.840486 2562 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 07 11:11:15.879225 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.840489 2562 flags.go:64] FLAG: --seccomp-default="false" Jul 07 11:11:15.879225 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.840491 2562 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 07 11:11:15.879225 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.840494 2562 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 07 11:11:15.880289 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.840497 2562 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 07 11:11:15.880289 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.840500 2562 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 07 11:11:15.880289 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.840503 2562 flags.go:64] FLAG: --storage-driver-password="root" Jul 07 11:11:15.880289 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.840506 2562 flags.go:64] FLAG: --storage-driver-secure="false" Jul 07 11:11:15.880289 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.840510 2562 flags.go:64] FLAG: --storage-driver-table="stats" Jul 07 11:11:15.880289 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.840513 2562 flags.go:64] FLAG: --storage-driver-user="root" Jul 07 11:11:15.880289 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.840516 2562 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 07 11:11:15.880289 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.840519 2562 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 07 11:11:15.880289 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.840522 2562 flags.go:64] FLAG: --system-cgroups="" Jul 07 11:11:15.880289 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.840525 2562 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 07 11:11:15.880289 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.840530 2562 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 07 11:11:15.880289 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.840533 2562 flags.go:64] FLAG: --tls-cert-file="" Jul 07 11:11:15.880289 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.840536 2562 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 07 11:11:15.880289 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.840540 2562 flags.go:64] FLAG: --tls-min-version="" Jul 07 11:11:15.880289 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.840543 2562 flags.go:64] FLAG: --tls-private-key-file="" Jul 07 11:11:15.880289 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.840545 2562 flags.go:64] FLAG: --topology-manager-policy="none" Jul 07 11:11:15.880289 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.840548 2562 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 07 11:11:15.880289 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.840551 2562 flags.go:64] FLAG: --topology-manager-scope="container" Jul 07 11:11:15.880289 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.840554 2562 flags.go:64] FLAG: --v="2" Jul 07 11:11:15.880289 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.840558 2562 flags.go:64] FLAG: --version="false" Jul 07 11:11:15.880289 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.840562 2562 flags.go:64] FLAG: --vmodule="" Jul 07 11:11:15.880289 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.840565 2562 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 07 11:11:15.880289 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.840568 2562 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 07 11:11:15.880289 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.840706 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 11:11:15.880289 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.840711 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 11:11:15.881141 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.840714 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 11:11:15.881141 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.840717 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 11:11:15.881141 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.840720 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 11:11:15.881141 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.840726 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 11:11:15.881141 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.840729 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 11:11:15.881141 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.840732 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 11:11:15.881141 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.840735 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 11:11:15.881141 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.840737 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 11:11:15.881141 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.840740 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 11:11:15.881141 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.840742 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 11:11:15.881141 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.840745 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 11:11:15.881141 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.840749 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 11:11:15.881141 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.840751 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 11:11:15.881141 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.840754 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 11:11:15.881141 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.840756 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 11:11:15.881141 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.840759 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 11:11:15.881141 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.840762 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 11:11:15.881141 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.840764 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 11:11:15.881141 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.840767 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 11:11:15.882202 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.840770 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 11:11:15.882202 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.840772 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 11:11:15.882202 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.840775 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 11:11:15.882202 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.840777 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 11:11:15.882202 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.840780 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 11:11:15.882202 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.840783 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 11:11:15.882202 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.840785 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 11:11:15.882202 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.840788 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 11:11:15.882202 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.840790 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 11:11:15.882202 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.840793 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 11:11:15.882202 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.840795 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 11:11:15.882202 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.840798 2562 feature_gate.go:328] unrecognized feature gate: Example Jul 07 11:11:15.882202 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.840801 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 11:11:15.882202 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.840803 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 11:11:15.882202 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.840805 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 11:11:15.882202 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.840808 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 11:11:15.882202 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.840812 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 11:11:15.882202 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.840815 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 11:11:15.882202 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.840817 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 11:11:15.882870 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.840820 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 11:11:15.882870 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.840822 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 11:11:15.882870 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.840825 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 11:11:15.882870 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.840827 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 11:11:15.882870 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.840830 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 11:11:15.882870 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.840833 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 11:11:15.882870 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.840836 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 11:11:15.882870 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.840839 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 11:11:15.882870 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.840841 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 11:11:15.882870 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.840844 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 11:11:15.882870 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.840846 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 11:11:15.882870 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.840849 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 11:11:15.882870 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.840852 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 11:11:15.882870 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.840854 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 11:11:15.882870 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.840859 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 11:11:15.882870 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.840862 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 11:11:15.882870 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.840865 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 11:11:15.882870 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.840868 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 11:11:15.882870 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.840870 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 11:11:15.882870 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.840873 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 11:11:15.885861 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.840876 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 11:11:15.885861 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.840879 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 11:11:15.885861 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.840881 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 11:11:15.885861 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.840885 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 11:11:15.885861 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.840889 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 11:11:15.885861 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.840892 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 11:11:15.885861 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.840894 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 11:11:15.885861 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.840897 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 11:11:15.885861 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.840899 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 11:11:15.885861 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.840903 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 11:11:15.885861 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.840906 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 11:11:15.885861 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.840908 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 11:11:15.885861 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.840926 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 11:11:15.885861 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.840930 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 11:11:15.885861 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.840933 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 11:11:15.885861 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.840936 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 11:11:15.885861 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.840939 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 11:11:15.885861 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.840943 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 11:11:15.885861 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.840946 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 11:11:15.886485 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.840948 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 11:11:15.886485 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.840951 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 11:11:15.886485 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.840953 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 11:11:15.886485 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.840956 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 11:11:15.886485 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.840958 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 11:11:15.886485 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.840961 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 11:11:15.886485 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.840964 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 11:11:15.886485 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.840966 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 11:11:15.886485 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.840969 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 11:11:15.886485 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.840971 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 11:11:15.886485 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.840974 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 11:11:15.886485 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.840976 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 11:11:15.886485 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.840979 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 11:11:15.886485 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.840981 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 11:11:15.886485 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.840983 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 11:11:15.886485 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.840986 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 11:11:15.886485 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.840988 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 11:11:15.886485 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.840991 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 11:11:15.886485 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.840993 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 11:11:15.886485 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.840996 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 11:11:15.942517 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.841617 2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 07 11:11:15.942517 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.847719 2562 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 07 11:11:15.942517 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.847733 2562 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 07 11:11:15.942517 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.848447 2562 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 07 11:11:15.942517 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.848461 2562 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 07 11:11:15.942517 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.848539 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 11:11:15.942517 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.848546 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 11:11:15.942517 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.848551 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 11:11:15.942517 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.848556 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 11:11:15.942517 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.848560 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 11:11:15.942517 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.848564 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 11:11:15.942517 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.848570 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 11:11:15.942517 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.848576 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 11:11:15.942517 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.848581 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 11:11:15.942517 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.848585 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 11:11:15.905906 ip-10-0-136-15 systemd[1]: Started Kubernetes Kubelet. Jul 07 11:11:15.943175 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.848589 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 11:11:15.943175 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.848593 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 11:11:15.943175 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.848598 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 11:11:15.943175 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.848602 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 11:11:15.943175 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.848606 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 11:11:15.943175 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.848610 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 11:11:15.943175 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.848614 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 11:11:15.943175 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.848618 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 11:11:15.943175 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.848623 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 11:11:15.943175 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.848631 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 11:11:15.943175 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.848636 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 11:11:15.943175 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.848640 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 11:11:15.943175 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.848644 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 11:11:15.943175 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.848648 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 11:11:15.943175 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.848652 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 11:11:15.943175 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.848656 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 11:11:15.943175 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.848660 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 11:11:15.943175 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.848663 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 11:11:15.943175 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.848667 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 11:11:15.943175 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.848671 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 11:11:15.943764 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.848674 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 11:11:15.943764 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.848678 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 11:11:15.943764 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.848682 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 11:11:15.943764 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.848686 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 11:11:15.943764 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.848690 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 11:11:15.943764 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.848694 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 11:11:15.943764 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.848698 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 11:11:15.943764 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.848704 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 11:11:15.943764 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.848708 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 11:11:15.943764 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.848712 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 11:11:15.943764 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.848715 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 11:11:15.943764 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.848719 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 11:11:15.943764 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.848723 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 11:11:15.943764 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.848727 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 11:11:15.943764 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.848730 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 11:11:15.943764 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.848735 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 11:11:15.943764 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.848738 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 11:11:15.943764 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.848742 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 11:11:15.943764 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.848746 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 11:11:15.943764 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.848750 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 11:11:15.944420 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.848754 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 11:11:15.944420 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.848760 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 11:11:15.944420 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.848763 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 11:11:15.944420 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.848767 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 11:11:15.944420 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.848771 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 11:11:15.944420 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.848775 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 11:11:15.944420 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.848779 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 11:11:15.944420 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.848783 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 11:11:15.944420 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.848786 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 11:11:15.944420 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.848791 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 11:11:15.944420 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.848795 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 11:11:15.944420 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.848799 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 11:11:15.944420 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.848803 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 11:11:15.944420 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.848807 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 11:11:15.944420 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.848811 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 11:11:15.944420 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.848815 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 11:11:15.944420 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.848819 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 11:11:15.944420 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.848823 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 11:11:15.944420 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.848827 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 11:11:15.944420 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.848831 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 11:11:15.945053 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.848834 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 11:11:15.945053 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.848838 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 11:11:15.945053 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.848842 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 11:11:15.945053 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.848846 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 11:11:15.945053 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.848850 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 11:11:15.945053 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.848853 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 11:11:15.945053 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.848857 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 11:11:15.945053 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.848862 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 11:11:15.945053 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.848867 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 11:11:15.945053 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.848871 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 11:11:15.945053 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.848877 2562 feature_gate.go:328] unrecognized feature gate: Example Jul 07 11:11:15.945053 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.848881 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 11:11:15.945053 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.848885 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 11:11:15.945053 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.848891 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 11:11:15.945053 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.848896 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 11:11:15.945053 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.848900 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 11:11:15.945053 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.848904 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 11:11:15.945053 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.848908 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 11:11:15.945053 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.848937 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 11:11:15.945861 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.848942 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 11:11:15.945861 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.848947 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 11:11:15.945861 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.848952 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 11:11:15.945861 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.848956 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 11:11:15.945861 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.848960 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 11:11:15.945861 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.848965 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 11:11:15.945861 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.848968 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 11:11:15.945861 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.848974 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 11:11:15.945861 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.848978 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 11:11:15.945861 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.848982 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 11:11:15.945861 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.848990 2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 07 11:11:15.945861 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.849145 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 11:11:15.945861 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.849151 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 11:11:15.945861 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.849154 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 11:11:15.945861 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.849158 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 11:11:15.946373 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.849162 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 11:11:15.946373 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.849165 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 11:11:15.946373 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.849169 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 11:11:15.946373 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.849172 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 11:11:15.946373 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.849176 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 11:11:15.946373 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.849179 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 11:11:15.946373 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.849182 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 11:11:15.946373 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.849186 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 11:11:15.946373 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.849191 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 11:11:15.946373 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.849194 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 11:11:15.946373 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.849198 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 11:11:15.946373 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.849201 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 11:11:15.946373 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.849206 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 11:11:15.946373 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.849209 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 11:11:15.946373 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.849212 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 11:11:15.946373 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.849215 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 11:11:15.946373 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.849219 2562 feature_gate.go:328] unrecognized feature gate: Example Jul 07 11:11:15.946373 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.849222 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 11:11:15.946373 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.849225 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 11:11:15.946373 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.849228 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 11:11:15.947090 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.849232 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 11:11:15.947090 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.849235 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 11:11:15.947090 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.849238 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 11:11:15.947090 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.849242 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 11:11:15.947090 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.849245 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 11:11:15.947090 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.849248 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 11:11:15.947090 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.849250 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 11:11:15.947090 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.849252 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 11:11:15.947090 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.849255 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 11:11:15.947090 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.849258 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 11:11:15.947090 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.849261 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 11:11:15.947090 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.849263 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 11:11:15.947090 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.849266 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 11:11:15.947090 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.849269 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 11:11:15.947090 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.849273 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 11:11:15.947090 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.849276 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 11:11:15.947090 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.849278 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 11:11:15.947090 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.849281 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 11:11:15.947090 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.849284 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 11:11:15.947744 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.849286 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 11:11:15.947744 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.849291 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 11:11:15.947744 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.849294 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 11:11:15.947744 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.849296 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 11:11:15.947744 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.849299 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 11:11:15.947744 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.849302 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 11:11:15.947744 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.849305 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 11:11:15.947744 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.849307 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 11:11:15.947744 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.849310 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 11:11:15.947744 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.849313 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 11:11:15.947744 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.849315 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 11:11:15.947744 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.849318 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 11:11:15.947744 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.849320 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 11:11:15.947744 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.849323 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 11:11:15.947744 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.849325 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 11:11:15.947744 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.849328 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 11:11:15.947744 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.849330 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 11:11:15.947744 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.849333 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 11:11:15.947744 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.849335 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 11:11:15.947744 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.849338 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 11:11:15.948410 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.849340 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 11:11:15.948410 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.849343 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 11:11:15.948410 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.849345 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 11:11:15.948410 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.849348 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 11:11:15.948410 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.849350 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 11:11:15.948410 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.849353 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 11:11:15.948410 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.849357 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 11:11:15.948410 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.849360 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 11:11:15.948410 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.849363 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 11:11:15.948410 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.849366 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 11:11:15.948410 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.849370 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 11:11:15.948410 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.849373 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 11:11:15.948410 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.849376 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 11:11:15.948410 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.849380 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 11:11:15.948410 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.849383 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 11:11:15.948410 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.849385 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 11:11:15.948410 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.849388 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 11:11:15.948410 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.849392 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 11:11:15.948410 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.849395 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 11:11:15.949399 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.849398 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 11:11:15.949399 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.849400 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 11:11:15.949399 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.849403 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 11:11:15.949399 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.849405 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 11:11:15.949399 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.849408 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 11:11:15.949399 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.849410 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 11:11:15.949399 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.849413 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 11:11:15.949399 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.849416 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 11:11:15.949399 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.849418 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 11:11:15.949399 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.849421 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 11:11:15.949399 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.849423 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 11:11:15.949399 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.849426 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 11:11:15.949399 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.849428 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 11:11:15.949399 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.849430 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 11:11:15.949399 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.849433 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 11:11:15.949399 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.849436 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 11:11:15.949399 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:15.849438 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 11:11:15.950424 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.849443 2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 07 11:11:15.950424 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.849538 2562 server.go:962] "Client rotation is on, will bootstrap in background" Jul 07 11:11:15.950424 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.851633 2562 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 07 11:11:15.950424 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.852940 2562 server.go:1019] "Starting client certificate rotation" Jul 07 11:11:15.950424 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.853042 2562 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 07 11:11:15.950424 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.853095 2562 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 07 11:11:15.950424 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.856763 2562 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 07 11:11:15.950424 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.858840 2562 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 07 11:11:15.950424 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.869016 2562 log.go:25] "Validated CRI v1 runtime API" Jul 07 11:11:15.950424 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.872806 2562 log.go:25] "Validated CRI v1 image API" Jul 07 11:11:15.950424 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.874153 2562 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 07 11:11:15.950424 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.879826 2562 fs.go:135] Filesystem UUIDs: map[2929eddb-b45e-48c7-8f6b-dc71ee710b5f:/dev/nvme0n1p3 38bf922f-4388-4c64-912f-f2e43245bf06:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2] Jul 07 11:11:15.950821 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.879839 2562 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 07 11:11:15.951240 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.884472 2562 manager.go:216] Machine: {Timestamp:2026-07-07 11:11:15.883315085 +0000 UTC m=+0.297184242 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3098156 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2bf0252cb2c7840efcd536b9f7eb1d SystemUUID:ec2bf025-2cb2-c784-0efc-d536b9f7eb1d BootID:e0dc3daa-ae00-4ecf-851d-a0849002072d Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:c7:37:1b:ca:6d Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:c7:37:1b:ca:6d Speed:0 Mtu:9001} {Name:ovs-system MacAddress:c2:70:b8:97:41:c4 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 07 11:11:15.951240 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.884618 2562 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 07 11:11:15.951240 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.884624 2562 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 07 11:11:15.951240 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.884809 2562 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 07 11:11:15.951240 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.887511 2562 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 07 11:11:15.951240 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.887543 2562 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-136-15.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 07 11:11:15.951240 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.888672 2562 topology_manager.go:138] "Creating topology manager with none policy" Jul 07 11:11:15.951661 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.888685 2562 container_manager_linux.go:309] "Creating device plugin manager" Jul 07 11:11:15.951661 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.888694 2562 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 07 11:11:15.951661 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.888711 2562 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 07 11:11:15.951661 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.889992 2562 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 07 11:11:15.951661 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.893015 2562 state_mem.go:36] "Initialized new in-memory state store" Jul 07 11:11:15.951661 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.893135 2562 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 07 11:11:15.951661 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.893160 2562 kubelet.go:486] "Attempting to sync node with API server" Jul 07 11:11:15.951661 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.893170 2562 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 07 11:11:15.951661 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.893179 2562 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 07 11:11:15.951661 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.893188 2562 kubelet.go:398] "Adding apiserver pod source" Jul 07 11:11:15.951661 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.893197 2562 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 07 11:11:15.951661 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.893489 2562 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 07 11:11:15.951661 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.893506 2562 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 07 11:11:15.951661 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.896538 2562 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 07 11:11:15.951661 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.899670 2562 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 07 11:11:15.951661 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.899716 2562 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 07 11:11:15.951661 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.901612 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 07 11:11:15.951661 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.901627 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 07 11:11:15.951661 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.901633 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 07 11:11:15.951661 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.901639 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 07 11:11:15.951661 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.901647 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 07 11:11:15.951661 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.901653 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 07 11:11:15.951661 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.901661 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 07 11:11:15.951661 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.901666 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 07 11:11:15.951661 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.901673 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 07 11:11:15.951661 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.901679 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 07 11:11:15.951661 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.901686 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 07 11:11:15.951661 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.901692 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 07 11:11:15.951661 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.902304 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 07 11:11:15.951661 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.902311 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 07 11:11:15.951661 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.904361 2562 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-136-15.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 07 11:11:15.951661 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:15.904470 2562 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 07 11:11:15.951661 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:15.904654 2562 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-136-15.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 07 11:11:15.951661 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.905130 2562 server.go:1268] "Started kubelet" Jul 07 11:11:15.951661 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.905218 2562 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 07 11:11:15.951661 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.905210 2562 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 07 11:11:15.951661 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.905273 2562 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 07 11:11:15.951661 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.907128 2562 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 07 11:11:15.951661 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.907221 2562 server.go:310] "Adding debug handlers to kubelet server" Jul 07 11:11:15.951661 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:15.910074 2562 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 07 11:11:15.952956 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.911659 2562 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-ctbb5" Jul 07 11:11:15.952956 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:15.911902 2562 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-15.ec2.internal.18bffd7989b34c2d default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-15.ec2.internal,UID:ip-10-0-136-15.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-136-15.ec2.internal,},FirstTimestamp:2026-07-07 11:11:15.905104941 +0000 UTC m=+0.318974100,LastTimestamp:2026-07-07 11:11:15.905104941 +0000 UTC m=+0.318974100,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-15.ec2.internal,}" Jul 07 11:11:15.952956 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.914543 2562 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 07 11:11:15.952956 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.914553 2562 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 07 11:11:15.952956 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:15.914597 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-15.ec2.internal\" not found" Jul 07 11:11:15.952956 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.915247 2562 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 07 11:11:15.952956 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.916161 2562 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 07 11:11:15.952956 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.916176 2562 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 07 11:11:15.952956 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.916347 2562 reconstruct.go:97] "Volume reconstruction finished" Jul 07 11:11:15.952956 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.916367 2562 reconciler.go:29] "Reconciler: start to sync state" Jul 07 11:11:15.952956 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.916580 2562 factory.go:55] Registering systemd factory Jul 07 11:11:15.952956 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.916596 2562 factory.go:223] Registration of the systemd container factory successfully Jul 07 11:11:15.952956 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:15.916701 2562 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-136-15.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 07 11:11:15.952956 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:15.916714 2562 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 07 11:11:15.952956 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.916800 2562 factory.go:153] Registering CRI-O factory Jul 07 11:11:15.952956 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.916812 2562 factory.go:223] Registration of the crio container factory successfully Jul 07 11:11:15.952956 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.916860 2562 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 07 11:11:15.952956 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.916880 2562 factory.go:103] Registering Raw factory Jul 07 11:11:15.952956 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.916892 2562 manager.go:1196] Started watching for new ooms in manager Jul 07 11:11:15.952956 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.917303 2562 manager.go:349] Starting recovery of all containers Jul 07 11:11:15.952956 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.917832 2562 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-ctbb5" Jul 07 11:11:15.952956 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.929471 2562 manager.go:354] Recovery completed Jul 07 11:11:15.952956 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.934581 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 11:11:15.952956 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.936343 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasSufficientMemory" Jul 07 11:11:15.952956 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.936365 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 11:11:15.952956 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.936374 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasSufficientPID" Jul 07 11:11:15.952956 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.936981 2562 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 07 11:11:15.952956 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.936988 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 11:11:15.952956 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.936992 2562 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 07 11:11:15.952956 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.937007 2562 state_mem.go:36] "Initialized new in-memory state store" Jul 07 11:11:15.952956 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.937665 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasSufficientMemory" Jul 07 11:11:15.952956 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.937690 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 11:11:15.953984 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.937699 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasSufficientPID" Jul 07 11:11:15.953984 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.943146 2562 policy_none.go:49] "None policy: Start" Jul 07 11:11:15.953984 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.943160 2562 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 07 11:11:15.953984 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.943170 2562 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 07 11:11:15.953984 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:15.944125 2562 policy_none.go:47] "Start" Jul 07 11:11:16.752928 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:16.706224 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-15.ec2.internal\" not found" Jul 07 11:11:16.752928 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:16.723489 2562 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-136-15.ec2.internal\" not found" node="ip-10-0-136-15.ec2.internal" Jul 07 11:11:16.752928 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:16.725705 2562 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 11:11:16.874490 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:16.800178 2562 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 11:11:16.874490 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:16.807299 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-15.ec2.internal\" not found" Jul 07 11:11:16.874490 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:16.853586 2562 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 07 11:11:16.874490 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:16.853684 2562 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 07 11:11:16.874490 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:16.853746 2562 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 07 11:11:16.895942 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:16.894134 2562 apiserver.go:52] "Watching apiserver" Jul 07 11:11:17.000880 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:16.906453 2562 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 07 11:11:17.000880 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:16.907759 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-15.ec2.internal\" not found" Jul 07 11:11:17.000880 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:16.914511 2562 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-136-15.ec2.internal" not found Jul 07 11:11:17.000880 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:16.920032 2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-06 11:06:16 +0000 UTC" deadline="2027-12-13 14:23:51.391593274 +0000 UTC" Jul 07 11:11:17.000880 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:16.920051 2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12579h12m34.471545014s" Jul 07 11:11:17.000880 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:16.931597 2562 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-136-15.ec2.internal" not found Jul 07 11:11:17.000880 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:16.990831 2562 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-136-15.ec2.internal" not found Jul 07 11:11:17.009259 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:17.008033 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-15.ec2.internal\" not found" Jul 07 11:11:17.134264 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.045699 2562 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 11:11:17.134264 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:17.108388 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-15.ec2.internal\" not found" Jul 07 11:11:17.265038 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:17.209107 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-15.ec2.internal\" not found" Jul 07 11:11:17.265038 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.259947 2562 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-136-15.ec2.internal" not found Jul 07 11:11:17.265038 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:17.259962 2562 csi_plugin.go:399] Failed to initialize CSINode: error updating CSINode annotation: timed out waiting for the condition; caused by: nodes "ip-10-0-136-15.ec2.internal" not found Jul 07 11:11:17.399433 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.298501 2562 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-136-15.ec2.internal" not found Jul 07 11:11:17.399433 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:17.309455 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-15.ec2.internal\" not found" Jul 07 11:11:17.399433 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.314761 2562 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-136-15.ec2.internal" not found Jul 07 11:11:17.399433 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.376307 2562 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-136-15.ec2.internal" not found Jul 07 11:11:17.516592 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:17.410316 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-15.ec2.internal\" not found" Jul 07 11:11:17.516592 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:17.511011 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-15.ec2.internal\" not found" Jul 07 11:11:17.611604 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:17.611553 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-15.ec2.internal\" not found" Jul 07 11:11:17.644683 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.644482 2562 manager.go:337] "Starting Device Plugin manager" Jul 07 11:11:17.644790 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:17.644705 2562 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 07 11:11:17.644790 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.644718 2562 server.go:87] "Starting device plugin registration server" Jul 07 11:11:17.644950 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.644939 2562 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 07 11:11:17.644986 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.644952 2562 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 07 11:11:17.645037 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.645021 2562 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 07 11:11:17.645136 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.645083 2562 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 07 11:11:17.645136 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.645094 2562 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 07 11:11:17.645692 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:17.645672 2562 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 07 11:11:17.645785 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:17.645726 2562 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-136-15.ec2.internal\" not found" Jul 07 11:11:17.646949 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.646936 2562 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-136-15.ec2.internal" not found Jul 07 11:11:17.647042 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:17.646953 2562 csi_plugin.go:399] Failed to initialize CSINode: error updating CSINode annotation: timed out waiting for the condition; caused by: nodes "ip-10-0-136-15.ec2.internal" not found Jul 07 11:11:17.746431 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.746401 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 11:11:17.748269 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.748247 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasSufficientMemory" Jul 07 11:11:17.748350 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.748289 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 11:11:17.748350 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.748300 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasSufficientPID" Jul 07 11:11:17.748350 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.748326 2562 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-136-15.ec2.internal" Jul 07 11:11:17.757816 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.757798 2562 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-136-15.ec2.internal" Jul 07 11:11:17.757869 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:17.757822 2562 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-136-15.ec2.internal\": node \"ip-10-0-136-15.ec2.internal\" not found" Jul 07 11:11:17.759107 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.759060 2562 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 07 11:11:17.760427 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.760410 2562 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 07 11:11:17.760484 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.760447 2562 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 07 11:11:17.760484 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.760468 2562 kubelet.go:2448] "Starting kubelet main sync loop" Jul 07 11:11:17.760573 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:17.760500 2562 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 07 11:11:17.762823 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.762794 2562 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 11:11:17.819621 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:17.819593 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-15.ec2.internal\" not found" Jul 07 11:11:17.861453 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.861430 2562 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-15.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-136-15.ec2.internal"] Jul 07 11:11:17.861516 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.861498 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 11:11:17.863190 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.863146 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasSufficientMemory" Jul 07 11:11:17.863190 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.863177 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 11:11:17.863190 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.863188 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasSufficientPID" Jul 07 11:11:17.864321 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.864309 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 11:11:17.864452 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.864435 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-15.ec2.internal" Jul 07 11:11:17.864490 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.864473 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 11:11:17.865042 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.865022 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasSufficientMemory" Jul 07 11:11:17.865042 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.865038 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasSufficientMemory" Jul 07 11:11:17.865256 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.865057 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 11:11:17.865256 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.865063 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 11:11:17.865256 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.865075 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasSufficientPID" Jul 07 11:11:17.865256 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.865085 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasSufficientPID" Jul 07 11:11:17.868404 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.868386 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=[] Jul 07 11:11:17.868404 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.868405 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-node-tuning-operator/tuned-vhcmx"] Jul 07 11:11:17.868527 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.868456 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 11:11:17.868527 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.868508 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-15.ec2.internal" Jul 07 11:11:17.868600 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.868537 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 11:11:17.869186 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.869170 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasSufficientMemory" Jul 07 11:11:17.869186 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.869178 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasSufficientMemory" Jul 07 11:11:17.869375 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.869200 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 11:11:17.869375 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.869217 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasSufficientPID" Jul 07 11:11:17.869375 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.869201 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 11:11:17.869375 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.869255 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasSufficientPID" Jul 07 11:11:17.870356 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.870341 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-additional-cni-plugins-6h8jc"] Jul 07 11:11:17.870460 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.870446 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-vhcmx" Jul 07 11:11:17.870504 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.870468 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 11:11:17.870504 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.870477 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 11:11:17.871231 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.871189 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasSufficientMemory" Jul 07 11:11:17.871231 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.871215 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 11:11:17.871231 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.871231 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasSufficientPID" Jul 07 11:11:17.871976 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.871711 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasSufficientMemory" Jul 07 11:11:17.871976 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.871775 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 11:11:17.871976 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.871809 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasSufficientPID" Jul 07 11:11:17.874429 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.874410 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/network-metrics-daemon-2q46z"] Jul 07 11:11:17.874522 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.874486 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 11:11:17.874797 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.874563 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-6h8jc" Jul 07 11:11:17.874797 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.874603 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 11:11:17.875314 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.875295 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-wq59m\"" Jul 07 11:11:17.875409 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.875348 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasSufficientMemory" Jul 07 11:11:17.875409 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.875372 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 11:11:17.875409 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.875383 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasSufficientPID" Jul 07 11:11:17.875564 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.875349 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasSufficientMemory" Jul 07 11:11:17.875564 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.875449 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 11:11:17.875564 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.875466 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasSufficientPID" Jul 07 11:11:17.875564 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.875545 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 07 11:11:17.875735 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.875678 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 07 11:11:17.876696 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.876681 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/node-ca-528n2"] Jul 07 11:11:17.876778 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.876766 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 11:11:17.876816 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.876788 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2q46z" Jul 07 11:11:17.876849 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.876815 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 11:11:17.877540 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.877521 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasSufficientMemory" Jul 07 11:11:17.877616 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.877548 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 11:11:17.877616 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.877558 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasSufficientPID" Jul 07 11:11:17.877616 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.877532 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasSufficientMemory" Jul 07 11:11:17.877616 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.877580 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 11:11:17.877616 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.877590 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasSufficientPID" Jul 07 11:11:17.877616 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:17.877603 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2q46z" podUID="94c776b2-5ce9-4210-adf5-31efabcef2af" Jul 07 11:11:17.878724 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.878708 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-4b5qw"] Jul 07 11:11:17.878810 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.878800 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 11:11:17.878810 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.878806 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-528n2" Jul 07 11:11:17.878874 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.878841 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 11:11:17.879702 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.879685 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasSufficientMemory" Jul 07 11:11:17.879789 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.879716 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 11:11:17.879789 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.879732 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasSufficientPID" Jul 07 11:11:17.880533 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.880517 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasSufficientMemory" Jul 07 11:11:17.880604 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.880546 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 11:11:17.880604 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.880566 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasSufficientPID" Jul 07 11:11:17.880669 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.880635 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 07 11:11:17.880805 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.880778 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 07 11:11:17.880907 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.880835 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-trhvm"] Jul 07 11:11:17.881002 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.880987 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-4b5qw" Jul 07 11:11:17.881063 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.881012 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 11:11:17.881063 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.881056 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 11:11:17.881494 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.881475 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 07 11:11:17.881728 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.881708 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-qqpsk\"" Jul 07 11:11:17.881820 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.881791 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasSufficientMemory" Jul 07 11:11:17.881875 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.881808 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasSufficientMemory" Jul 07 11:11:17.881875 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.881815 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 11:11:17.881875 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.881840 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 11:11:17.881875 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.881856 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasSufficientPID" Jul 07 11:11:17.881875 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.881856 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasSufficientPID" Jul 07 11:11:17.883098 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.883084 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-trhvm" Jul 07 11:11:17.883137 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.883108 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 11:11:17.883754 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.883738 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasSufficientMemory" Jul 07 11:11:17.883801 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.883768 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 11:11:17.883801 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.883779 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasSufficientPID" Jul 07 11:11:17.884954 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.884937 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 07 11:11:17.886435 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.886418 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 07 11:11:17.886517 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.886449 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 07 11:11:17.886577 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.886546 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 07 11:11:17.886678 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.886660 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-fs9bb\"" Jul 07 11:11:17.886782 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.886693 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-8fnm4\"" Jul 07 11:11:17.887625 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.887605 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 07 11:11:17.887625 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.887607 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 07 11:11:17.887738 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.887605 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 07 11:11:17.887738 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.887607 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 07 11:11:17.887738 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.887665 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-k672b\"" Jul 07 11:11:17.889775 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:17.889754 2562 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-15.ec2.internal\" not found" node="ip-10-0-136-15.ec2.internal" Jul 07 11:11:17.909515 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.909496 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ovn-kubernetes/ovnkube-node-tzf62"] Jul 07 11:11:17.909806 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.909791 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 11:11:17.910530 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.910501 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasSufficientMemory" Jul 07 11:11:17.910629 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.910539 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 11:11:17.910629 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.910549 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasSufficientPID" Jul 07 11:11:17.910847 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:17.910829 2562 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-15.ec2.internal\" not found" node="ip-10-0-136-15.ec2.internal" Jul 07 11:11:17.911915 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.911901 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n8tzm"] Jul 07 11:11:17.912057 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.912040 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-tzf62" Jul 07 11:11:17.912112 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.912079 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 11:11:17.912112 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.912083 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 11:11:17.912848 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.912822 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasSufficientMemory" Jul 07 11:11:17.912955 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.912881 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasSufficientMemory" Jul 07 11:11:17.912955 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.912891 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 11:11:17.912955 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.912905 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 11:11:17.912955 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.912911 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasSufficientPID" Jul 07 11:11:17.912955 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.912923 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasSufficientPID" Jul 07 11:11:17.914223 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.914207 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n8tzm" Jul 07 11:11:17.914321 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.914234 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 11:11:17.915019 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.914977 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasSufficientMemory" Jul 07 11:11:17.915019 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.915013 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 11:11:17.915184 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.915029 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasSufficientPID" Jul 07 11:11:17.916962 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.916946 2562 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 07 11:11:17.918866 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.918846 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 07 11:11:17.919205 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.919192 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 07 11:11:17.919408 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.919394 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 07 11:11:17.919408 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.919398 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 07 11:11:17.919683 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:17.919669 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-15.ec2.internal\" not found" Jul 07 11:11:17.919827 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.919812 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 07 11:11:17.919876 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.919830 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 07 11:11:17.919876 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.919818 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 07 11:11:17.919975 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.919942 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-n99rt\"" Jul 07 11:11:17.920173 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.920158 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 07 11:11:17.920219 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.920171 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-h59f8\"" Jul 07 11:11:17.920259 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.920231 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 07 11:11:17.957986 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.957962 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-operator/iptables-alerter-dvhgx"] Jul 07 11:11:17.958231 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.958213 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 11:11:17.959043 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.959027 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasSufficientMemory" Jul 07 11:11:17.959124 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.959051 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 11:11:17.959124 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.959062 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasSufficientPID" Jul 07 11:11:17.960536 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.960522 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-target-sc5bl"] Jul 07 11:11:17.960632 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.960618 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-dvhgx" Jul 07 11:11:17.960670 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.960645 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 11:11:17.960670 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.960653 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 11:11:17.961332 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.961315 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasSufficientMemory" Jul 07 11:11:17.961393 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.961342 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 11:11:17.961393 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.961353 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasSufficientPID" Jul 07 11:11:17.961494 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.961315 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasSufficientMemory" Jul 07 11:11:17.961494 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.961421 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 11:11:17.961494 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.961437 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasSufficientPID" Jul 07 11:11:17.962835 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.962819 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sc5bl" Jul 07 11:11:17.962883 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.962847 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 11:11:17.963489 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.963475 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasSufficientMemory" Jul 07 11:11:17.963554 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.963499 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 11:11:17.963554 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.963509 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasSufficientPID" Jul 07 11:11:17.963625 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:17.963562 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sc5bl" podUID="b9ca66a2-4c5a-4816-a081-bb0fecbdab5d" Jul 07 11:11:17.965025 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.965006 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 07 11:11:17.965098 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.965083 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-25crw\"" Jul 07 11:11:17.966222 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.966205 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 07 11:11:17.966318 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.966226 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 07 11:11:17.982207 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.982188 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-vrfk8"] Jul 07 11:11:17.982330 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.982320 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 11:11:17.983151 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.983134 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasSufficientMemory" Jul 07 11:11:17.983221 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.983165 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 11:11:17.983221 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.983176 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasSufficientPID" Jul 07 11:11:17.983319 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.983308 2562 predicate.go:218] "Predicate failed on Pod" pod="kube-system/global-pull-secret-syncer-vrfk8" err="Predicate NodeAffinity failed: node(s) didn't match Pod's node affinity/selector" Jul 07 11:11:17.983351 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.983321 2562 allocation_manager.go:702] "Pod admission denied" podUID="f5e35157-929c-493a-9da0-386b354a2acb" pod="kube-system/global-pull-secret-syncer-vrfk8" reason="NodeAffinity" message="Predicate NodeAffinity failed: node(s) didn't match Pod's node affinity/selector" Jul 07 11:11:17.986594 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.986577 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/node-resolver-77v22"] Jul 07 11:11:17.986725 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.986715 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 11:11:17.987497 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.987479 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasSufficientMemory" Jul 07 11:11:17.987577 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.987504 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 11:11:17.987577 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.987516 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasSufficientPID" Jul 07 11:11:17.989206 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.989192 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-77v22" Jul 07 11:11:17.989295 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.989217 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 11:11:17.989882 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.989865 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasSufficientMemory" Jul 07 11:11:17.989966 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.989893 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 11:11:17.989966 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.989905 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasSufficientPID" Jul 07 11:11:17.992612 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.992596 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 07 11:11:17.993032 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.993016 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-k9xpx\"" Jul 07 11:11:17.993207 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:17.993193 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 07 11:11:18.014475 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.014456 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/95dba470-f153-4c3c-93d6-c3fb757fec0b-etc-kubernetes\") pod \"multus-trhvm\" (UID: \"95dba470-f153-4c3c-93d6-c3fb757fec0b\") " pod="openshift-multus/multus-trhvm" Jul 07 11:11:18.014580 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.014480 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d2340b4b-179a-4c0a-9879-13b49897c939-ovnkube-config\") pod \"ovnkube-node-tzf62\" (UID: \"d2340b4b-179a-4c0a-9879-13b49897c939\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzf62" Jul 07 11:11:18.014580 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.014499 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kfm4r\" (UniqueName: \"kubernetes.io/projected/e93d8868-ca25-4c0a-9f25-da7b8c286f60-kube-api-access-kfm4r\") pod \"node-ca-528n2\" (UID: \"e93d8868-ca25-4c0a-9f25-da7b8c286f60\") " pod="openshift-image-registry/node-ca-528n2" Jul 07 11:11:18.014580 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.014514 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/95dba470-f153-4c3c-93d6-c3fb757fec0b-system-cni-dir\") pod \"multus-trhvm\" (UID: \"95dba470-f153-4c3c-93d6-c3fb757fec0b\") " pod="openshift-multus/multus-trhvm" Jul 07 11:11:18.014580 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.014533 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/95dba470-f153-4c3c-93d6-c3fb757fec0b-cnibin\") pod \"multus-trhvm\" (UID: \"95dba470-f153-4c3c-93d6-c3fb757fec0b\") " pod="openshift-multus/multus-trhvm" Jul 07 11:11:18.014775 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.014582 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d2340b4b-179a-4c0a-9879-13b49897c939-etc-openvswitch\") pod \"ovnkube-node-tzf62\" (UID: \"d2340b4b-179a-4c0a-9879-13b49897c939\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzf62" Jul 07 11:11:18.014775 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.014626 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d2340b4b-179a-4c0a-9879-13b49897c939-node-log\") pod \"ovnkube-node-tzf62\" (UID: \"d2340b4b-179a-4c0a-9879-13b49897c939\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzf62" Jul 07 11:11:18.014775 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.014657 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d2340b4b-179a-4c0a-9879-13b49897c939-systemd-units\") pod \"ovnkube-node-tzf62\" (UID: \"d2340b4b-179a-4c0a-9879-13b49897c939\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzf62" Jul 07 11:11:18.014775 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.014699 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/95dba470-f153-4c3c-93d6-c3fb757fec0b-host-run-k8s-cni-cncf-io\") pod \"multus-trhvm\" (UID: \"95dba470-f153-4c3c-93d6-c3fb757fec0b\") " pod="openshift-multus/multus-trhvm" Jul 07 11:11:18.014775 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.014737 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/522a4c3d-a66a-4f5f-b8fa-e25cbf48c969-etc-sysctl-d\") pod \"tuned-vhcmx\" (UID: \"522a4c3d-a66a-4f5f-b8fa-e25cbf48c969\") " pod="openshift-cluster-node-tuning-operator/tuned-vhcmx" Jul 07 11:11:18.014775 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.014770 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/522a4c3d-a66a-4f5f-b8fa-e25cbf48c969-host\") pod \"tuned-vhcmx\" (UID: \"522a4c3d-a66a-4f5f-b8fa-e25cbf48c969\") " pod="openshift-cluster-node-tuning-operator/tuned-vhcmx" Jul 07 11:11:18.015057 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.014794 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d2340b4b-179a-4c0a-9879-13b49897c939-log-socket\") pod \"ovnkube-node-tzf62\" (UID: \"d2340b4b-179a-4c0a-9879-13b49897c939\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzf62" Jul 07 11:11:18.015057 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.014821 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/663d4847-eafe-43d6-a4b8-2bec15919119-konnectivity-ca\") pod \"konnectivity-agent-4b5qw\" (UID: \"663d4847-eafe-43d6-a4b8-2bec15919119\") " pod="kube-system/konnectivity-agent-4b5qw" Jul 07 11:11:18.015057 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.014845 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/95dba470-f153-4c3c-93d6-c3fb757fec0b-os-release\") pod \"multus-trhvm\" (UID: \"95dba470-f153-4c3c-93d6-c3fb757fec0b\") " pod="openshift-multus/multus-trhvm" Jul 07 11:11:18.015057 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.014869 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6rsn9\" (UniqueName: \"kubernetes.io/projected/95dba470-f153-4c3c-93d6-c3fb757fec0b-kube-api-access-6rsn9\") pod \"multus-trhvm\" (UID: \"95dba470-f153-4c3c-93d6-c3fb757fec0b\") " pod="openshift-multus/multus-trhvm" Jul 07 11:11:18.015057 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.014892 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d2340b4b-179a-4c0a-9879-13b49897c939-host-slash\") pod \"ovnkube-node-tzf62\" (UID: \"d2340b4b-179a-4c0a-9879-13b49897c939\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzf62" Jul 07 11:11:18.015057 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.014919 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d2340b4b-179a-4c0a-9879-13b49897c939-env-overrides\") pod \"ovnkube-node-tzf62\" (UID: \"d2340b4b-179a-4c0a-9879-13b49897c939\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzf62" Jul 07 11:11:18.015057 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.014945 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6cc36837-424c-44fd-9644-1628e5e5d144-system-cni-dir\") pod \"multus-additional-cni-plugins-6h8jc\" (UID: \"6cc36837-424c-44fd-9644-1628e5e5d144\") " pod="openshift-multus/multus-additional-cni-plugins-6h8jc" Jul 07 11:11:18.015057 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.014989 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6cc36837-424c-44fd-9644-1628e5e5d144-cni-binary-copy\") pod \"multus-additional-cni-plugins-6h8jc\" (UID: \"6cc36837-424c-44fd-9644-1628e5e5d144\") " pod="openshift-multus/multus-additional-cni-plugins-6h8jc" Jul 07 11:11:18.015057 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.015009 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/6cc36837-424c-44fd-9644-1628e5e5d144-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-6h8jc\" (UID: \"6cc36837-424c-44fd-9644-1628e5e5d144\") " pod="openshift-multus/multus-additional-cni-plugins-6h8jc" Jul 07 11:11:18.015057 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.015025 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/94c776b2-5ce9-4210-adf5-31efabcef2af-metrics-certs\") pod \"network-metrics-daemon-2q46z\" (UID: \"94c776b2-5ce9-4210-adf5-31efabcef2af\") " pod="openshift-multus/network-metrics-daemon-2q46z" Jul 07 11:11:18.015057 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.015039 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/95dba470-f153-4c3c-93d6-c3fb757fec0b-cni-binary-copy\") pod \"multus-trhvm\" (UID: \"95dba470-f153-4c3c-93d6-c3fb757fec0b\") " pod="openshift-multus/multus-trhvm" Jul 07 11:11:18.015546 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.015064 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/95dba470-f153-4c3c-93d6-c3fb757fec0b-host-var-lib-kubelet\") pod \"multus-trhvm\" (UID: \"95dba470-f153-4c3c-93d6-c3fb757fec0b\") " pod="openshift-multus/multus-trhvm" Jul 07 11:11:18.015546 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.015088 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/95dba470-f153-4c3c-93d6-c3fb757fec0b-host-run-multus-certs\") pod \"multus-trhvm\" (UID: \"95dba470-f153-4c3c-93d6-c3fb757fec0b\") " pod="openshift-multus/multus-trhvm" Jul 07 11:11:18.015546 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.015104 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d2340b4b-179a-4c0a-9879-13b49897c939-host-run-netns\") pod \"ovnkube-node-tzf62\" (UID: \"d2340b4b-179a-4c0a-9879-13b49897c939\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzf62" Jul 07 11:11:18.015546 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.015119 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d2340b4b-179a-4c0a-9879-13b49897c939-var-lib-openvswitch\") pod \"ovnkube-node-tzf62\" (UID: \"d2340b4b-179a-4c0a-9879-13b49897c939\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzf62" Jul 07 11:11:18.015546 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.015150 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/5e4c9eaf-157e-4ada-8ee8-3d9319eb26a9-kubelet-dir\") pod \"aws-ebs-csi-driver-node-n8tzm\" (UID: \"5e4c9eaf-157e-4ada-8ee8-3d9319eb26a9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n8tzm" Jul 07 11:11:18.015546 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.015182 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/522a4c3d-a66a-4f5f-b8fa-e25cbf48c969-tmp\") pod \"tuned-vhcmx\" (UID: \"522a4c3d-a66a-4f5f-b8fa-e25cbf48c969\") " pod="openshift-cluster-node-tuning-operator/tuned-vhcmx" Jul 07 11:11:18.015546 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.015206 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6cc36837-424c-44fd-9644-1628e5e5d144-tuning-conf-dir\") pod \"multus-additional-cni-plugins-6h8jc\" (UID: \"6cc36837-424c-44fd-9644-1628e5e5d144\") " pod="openshift-multus/multus-additional-cni-plugins-6h8jc" Jul 07 11:11:18.015546 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.015234 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/663d4847-eafe-43d6-a4b8-2bec15919119-agent-certs\") pod \"konnectivity-agent-4b5qw\" (UID: \"663d4847-eafe-43d6-a4b8-2bec15919119\") " pod="kube-system/konnectivity-agent-4b5qw" Jul 07 11:11:18.015546 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.015259 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/522a4c3d-a66a-4f5f-b8fa-e25cbf48c969-etc-sysconfig\") pod \"tuned-vhcmx\" (UID: \"522a4c3d-a66a-4f5f-b8fa-e25cbf48c969\") " pod="openshift-cluster-node-tuning-operator/tuned-vhcmx" Jul 07 11:11:18.015546 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.015283 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/95dba470-f153-4c3c-93d6-c3fb757fec0b-host-var-lib-cni-bin\") pod \"multus-trhvm\" (UID: \"95dba470-f153-4c3c-93d6-c3fb757fec0b\") " pod="openshift-multus/multus-trhvm" Jul 07 11:11:18.015546 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.015309 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/95dba470-f153-4c3c-93d6-c3fb757fec0b-multus-daemon-config\") pod \"multus-trhvm\" (UID: \"95dba470-f153-4c3c-93d6-c3fb757fec0b\") " pod="openshift-multus/multus-trhvm" Jul 07 11:11:18.015546 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.015333 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d2340b4b-179a-4c0a-9879-13b49897c939-run-systemd\") pod \"ovnkube-node-tzf62\" (UID: \"d2340b4b-179a-4c0a-9879-13b49897c939\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzf62" Jul 07 11:11:18.015546 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.015359 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d2340b4b-179a-4c0a-9879-13b49897c939-host-cni-netd\") pod \"ovnkube-node-tzf62\" (UID: \"d2340b4b-179a-4c0a-9879-13b49897c939\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzf62" Jul 07 11:11:18.015546 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.015383 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/5e4c9eaf-157e-4ada-8ee8-3d9319eb26a9-etc-selinux\") pod \"aws-ebs-csi-driver-node-n8tzm\" (UID: \"5e4c9eaf-157e-4ada-8ee8-3d9319eb26a9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n8tzm" Jul 07 11:11:18.016181 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.015408 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/522a4c3d-a66a-4f5f-b8fa-e25cbf48c969-var-lib-kubelet\") pod \"tuned-vhcmx\" (UID: \"522a4c3d-a66a-4f5f-b8fa-e25cbf48c969\") " pod="openshift-cluster-node-tuning-operator/tuned-vhcmx" Jul 07 11:11:18.016181 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.015431 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7hmhh\" (UniqueName: \"kubernetes.io/projected/522a4c3d-a66a-4f5f-b8fa-e25cbf48c969-kube-api-access-7hmhh\") pod \"tuned-vhcmx\" (UID: \"522a4c3d-a66a-4f5f-b8fa-e25cbf48c969\") " pod="openshift-cluster-node-tuning-operator/tuned-vhcmx" Jul 07 11:11:18.016181 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.015469 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/95dba470-f153-4c3c-93d6-c3fb757fec0b-host-run-netns\") pod \"multus-trhvm\" (UID: \"95dba470-f153-4c3c-93d6-c3fb757fec0b\") " pod="openshift-multus/multus-trhvm" Jul 07 11:11:18.016181 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.015515 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d2340b4b-179a-4c0a-9879-13b49897c939-run-openvswitch\") pod \"ovnkube-node-tzf62\" (UID: \"d2340b4b-179a-4c0a-9879-13b49897c939\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzf62" Jul 07 11:11:18.016181 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.015555 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/5e4c9eaf-157e-4ada-8ee8-3d9319eb26a9-socket-dir\") pod \"aws-ebs-csi-driver-node-n8tzm\" (UID: \"5e4c9eaf-157e-4ada-8ee8-3d9319eb26a9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n8tzm" Jul 07 11:11:18.016181 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.015591 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/5e4c9eaf-157e-4ada-8ee8-3d9319eb26a9-registration-dir\") pod \"aws-ebs-csi-driver-node-n8tzm\" (UID: \"5e4c9eaf-157e-4ada-8ee8-3d9319eb26a9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n8tzm" Jul 07 11:11:18.016181 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.015636 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/95dba470-f153-4c3c-93d6-c3fb757fec0b-multus-cni-dir\") pod \"multus-trhvm\" (UID: \"95dba470-f153-4c3c-93d6-c3fb757fec0b\") " pod="openshift-multus/multus-trhvm" Jul 07 11:11:18.016181 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.015666 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/95dba470-f153-4c3c-93d6-c3fb757fec0b-host-var-lib-cni-multus\") pod \"multus-trhvm\" (UID: \"95dba470-f153-4c3c-93d6-c3fb757fec0b\") " pod="openshift-multus/multus-trhvm" Jul 07 11:11:18.016181 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.015691 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/95dba470-f153-4c3c-93d6-c3fb757fec0b-multus-conf-dir\") pod \"multus-trhvm\" (UID: \"95dba470-f153-4c3c-93d6-c3fb757fec0b\") " pod="openshift-multus/multus-trhvm" Jul 07 11:11:18.016181 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.015727 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6cc36837-424c-44fd-9644-1628e5e5d144-os-release\") pod \"multus-additional-cni-plugins-6h8jc\" (UID: \"6cc36837-424c-44fd-9644-1628e5e5d144\") " pod="openshift-multus/multus-additional-cni-plugins-6h8jc" Jul 07 11:11:18.016181 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.015754 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4jfp9\" (UniqueName: \"kubernetes.io/projected/94c776b2-5ce9-4210-adf5-31efabcef2af-kube-api-access-4jfp9\") pod \"network-metrics-daemon-2q46z\" (UID: \"94c776b2-5ce9-4210-adf5-31efabcef2af\") " pod="openshift-multus/network-metrics-daemon-2q46z" Jul 07 11:11:18.016181 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.015769 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-644qj\" (UniqueName: \"kubernetes.io/projected/5e4c9eaf-157e-4ada-8ee8-3d9319eb26a9-kube-api-access-644qj\") pod \"aws-ebs-csi-driver-node-n8tzm\" (UID: \"5e4c9eaf-157e-4ada-8ee8-3d9319eb26a9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n8tzm" Jul 07 11:11:18.016181 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.015784 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/522a4c3d-a66a-4f5f-b8fa-e25cbf48c969-etc-kubernetes\") pod \"tuned-vhcmx\" (UID: \"522a4c3d-a66a-4f5f-b8fa-e25cbf48c969\") " pod="openshift-cluster-node-tuning-operator/tuned-vhcmx" Jul 07 11:11:18.016633 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.015807 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/95dba470-f153-4c3c-93d6-c3fb757fec0b-multus-socket-dir-parent\") pod \"multus-trhvm\" (UID: \"95dba470-f153-4c3c-93d6-c3fb757fec0b\") " pod="openshift-multus/multus-trhvm" Jul 07 11:11:18.016633 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.015863 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dcf962a5f27f6d615415b27111cc76dc-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-15.ec2.internal\" (UID: \"dcf962a5f27f6d615415b27111cc76dc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-15.ec2.internal" Jul 07 11:11:18.016633 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.015882 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6cc36837-424c-44fd-9644-1628e5e5d144-cnibin\") pod \"multus-additional-cni-plugins-6h8jc\" (UID: \"6cc36837-424c-44fd-9644-1628e5e5d144\") " pod="openshift-multus/multus-additional-cni-plugins-6h8jc" Jul 07 11:11:18.016633 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.015900 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/6cc36837-424c-44fd-9644-1628e5e5d144-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-6h8jc\" (UID: \"6cc36837-424c-44fd-9644-1628e5e5d144\") " pod="openshift-multus/multus-additional-cni-plugins-6h8jc" Jul 07 11:11:18.016633 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.015916 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ztsrh\" (UniqueName: \"kubernetes.io/projected/6cc36837-424c-44fd-9644-1628e5e5d144-kube-api-access-ztsrh\") pod \"multus-additional-cni-plugins-6h8jc\" (UID: \"6cc36837-424c-44fd-9644-1628e5e5d144\") " pod="openshift-multus/multus-additional-cni-plugins-6h8jc" Jul 07 11:11:18.016633 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.015963 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/e93d8868-ca25-4c0a-9f25-da7b8c286f60-serviceca\") pod \"node-ca-528n2\" (UID: \"e93d8868-ca25-4c0a-9f25-da7b8c286f60\") " pod="openshift-image-registry/node-ca-528n2" Jul 07 11:11:18.016633 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.016002 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/5e4c9eaf-157e-4ada-8ee8-3d9319eb26a9-device-dir\") pod \"aws-ebs-csi-driver-node-n8tzm\" (UID: \"5e4c9eaf-157e-4ada-8ee8-3d9319eb26a9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n8tzm" Jul 07 11:11:18.016633 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.016041 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/522a4c3d-a66a-4f5f-b8fa-e25cbf48c969-etc-sysctl-conf\") pod \"tuned-vhcmx\" (UID: \"522a4c3d-a66a-4f5f-b8fa-e25cbf48c969\") " pod="openshift-cluster-node-tuning-operator/tuned-vhcmx" Jul 07 11:11:18.016633 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.016070 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d2340b4b-179a-4c0a-9879-13b49897c939-host-run-ovn-kubernetes\") pod \"ovnkube-node-tzf62\" (UID: \"d2340b4b-179a-4c0a-9879-13b49897c939\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzf62" Jul 07 11:11:18.016633 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.016093 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tpn9m\" (UniqueName: \"kubernetes.io/projected/d2340b4b-179a-4c0a-9879-13b49897c939-kube-api-access-tpn9m\") pod \"ovnkube-node-tzf62\" (UID: \"d2340b4b-179a-4c0a-9879-13b49897c939\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzf62" Jul 07 11:11:18.016633 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.016127 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a0c235cfc97783f979ae1f6fa1e22e58-config\") pod \"kube-apiserver-proxy-ip-10-0-136-15.ec2.internal\" (UID: \"a0c235cfc97783f979ae1f6fa1e22e58\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-15.ec2.internal" Jul 07 11:11:18.016633 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.016147 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/5e4c9eaf-157e-4ada-8ee8-3d9319eb26a9-sys-fs\") pod \"aws-ebs-csi-driver-node-n8tzm\" (UID: \"5e4c9eaf-157e-4ada-8ee8-3d9319eb26a9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n8tzm" Jul 07 11:11:18.016633 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.016169 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/522a4c3d-a66a-4f5f-b8fa-e25cbf48c969-run\") pod \"tuned-vhcmx\" (UID: \"522a4c3d-a66a-4f5f-b8fa-e25cbf48c969\") " pod="openshift-cluster-node-tuning-operator/tuned-vhcmx" Jul 07 11:11:18.017105 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.016188 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/95dba470-f153-4c3c-93d6-c3fb757fec0b-hostroot\") pod \"multus-trhvm\" (UID: \"95dba470-f153-4c3c-93d6-c3fb757fec0b\") " pod="openshift-multus/multus-trhvm" Jul 07 11:11:18.017105 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.016204 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d2340b4b-179a-4c0a-9879-13b49897c939-run-ovn\") pod \"ovnkube-node-tzf62\" (UID: \"d2340b4b-179a-4c0a-9879-13b49897c939\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzf62" Jul 07 11:11:18.017105 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.016222 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d2340b4b-179a-4c0a-9879-13b49897c939-host-cni-bin\") pod \"ovnkube-node-tzf62\" (UID: \"d2340b4b-179a-4c0a-9879-13b49897c939\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzf62" Jul 07 11:11:18.017105 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.016245 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/522a4c3d-a66a-4f5f-b8fa-e25cbf48c969-etc-modprobe-d\") pod \"tuned-vhcmx\" (UID: \"522a4c3d-a66a-4f5f-b8fa-e25cbf48c969\") " pod="openshift-cluster-node-tuning-operator/tuned-vhcmx" Jul 07 11:11:18.017105 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.016286 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/522a4c3d-a66a-4f5f-b8fa-e25cbf48c969-lib-modules\") pod \"tuned-vhcmx\" (UID: \"522a4c3d-a66a-4f5f-b8fa-e25cbf48c969\") " pod="openshift-cluster-node-tuning-operator/tuned-vhcmx" Jul 07 11:11:18.017105 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.016304 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d2340b4b-179a-4c0a-9879-13b49897c939-host-kubelet\") pod \"ovnkube-node-tzf62\" (UID: \"d2340b4b-179a-4c0a-9879-13b49897c939\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzf62" Jul 07 11:11:18.017105 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.016340 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d2340b4b-179a-4c0a-9879-13b49897c939-ovn-node-metrics-cert\") pod \"ovnkube-node-tzf62\" (UID: \"d2340b4b-179a-4c0a-9879-13b49897c939\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzf62" Jul 07 11:11:18.017105 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.016362 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/522a4c3d-a66a-4f5f-b8fa-e25cbf48c969-etc-tuned\") pod \"tuned-vhcmx\" (UID: \"522a4c3d-a66a-4f5f-b8fa-e25cbf48c969\") " pod="openshift-cluster-node-tuning-operator/tuned-vhcmx" Jul 07 11:11:18.017105 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.016378 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d2340b4b-179a-4c0a-9879-13b49897c939-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-tzf62\" (UID: \"d2340b4b-179a-4c0a-9879-13b49897c939\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzf62" Jul 07 11:11:18.017105 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.016394 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d2340b4b-179a-4c0a-9879-13b49897c939-ovnkube-script-lib\") pod \"ovnkube-node-tzf62\" (UID: \"d2340b4b-179a-4c0a-9879-13b49897c939\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzf62" Jul 07 11:11:18.017105 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.016411 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/dcf962a5f27f6d615415b27111cc76dc-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-15.ec2.internal\" (UID: \"dcf962a5f27f6d615415b27111cc76dc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-15.ec2.internal" Jul 07 11:11:18.017105 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.016426 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e93d8868-ca25-4c0a-9f25-da7b8c286f60-host\") pod \"node-ca-528n2\" (UID: \"e93d8868-ca25-4c0a-9f25-da7b8c286f60\") " pod="openshift-image-registry/node-ca-528n2" Jul 07 11:11:18.017105 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.016440 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/522a4c3d-a66a-4f5f-b8fa-e25cbf48c969-etc-systemd\") pod \"tuned-vhcmx\" (UID: \"522a4c3d-a66a-4f5f-b8fa-e25cbf48c969\") " pod="openshift-cluster-node-tuning-operator/tuned-vhcmx" Jul 07 11:11:18.017105 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.016456 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/522a4c3d-a66a-4f5f-b8fa-e25cbf48c969-sys\") pod \"tuned-vhcmx\" (UID: \"522a4c3d-a66a-4f5f-b8fa-e25cbf48c969\") " pod="openshift-cluster-node-tuning-operator/tuned-vhcmx" Jul 07 11:11:18.020325 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:18.020310 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-15.ec2.internal\" not found" Jul 07 11:11:18.117006 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.116945 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d2340b4b-179a-4c0a-9879-13b49897c939-host-cni-netd\") pod \"ovnkube-node-tzf62\" (UID: \"d2340b4b-179a-4c0a-9879-13b49897c939\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzf62" Jul 07 11:11:18.117006 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.116977 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/5e4c9eaf-157e-4ada-8ee8-3d9319eb26a9-etc-selinux\") pod \"aws-ebs-csi-driver-node-n8tzm\" (UID: \"5e4c9eaf-157e-4ada-8ee8-3d9319eb26a9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n8tzm" Jul 07 11:11:18.117006 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.117002 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/522a4c3d-a66a-4f5f-b8fa-e25cbf48c969-var-lib-kubelet\") pod \"tuned-vhcmx\" (UID: \"522a4c3d-a66a-4f5f-b8fa-e25cbf48c969\") " pod="openshift-cluster-node-tuning-operator/tuned-vhcmx" Jul 07 11:11:18.117175 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.117121 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/5e4c9eaf-157e-4ada-8ee8-3d9319eb26a9-etc-selinux\") pod \"aws-ebs-csi-driver-node-n8tzm\" (UID: \"5e4c9eaf-157e-4ada-8ee8-3d9319eb26a9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n8tzm" Jul 07 11:11:18.117175 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.117131 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d2340b4b-179a-4c0a-9879-13b49897c939-host-cni-netd\") pod \"ovnkube-node-tzf62\" (UID: \"d2340b4b-179a-4c0a-9879-13b49897c939\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzf62" Jul 07 11:11:18.117254 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.117171 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7hmhh\" (UniqueName: \"kubernetes.io/projected/522a4c3d-a66a-4f5f-b8fa-e25cbf48c969-kube-api-access-7hmhh\") pod \"tuned-vhcmx\" (UID: \"522a4c3d-a66a-4f5f-b8fa-e25cbf48c969\") " pod="openshift-cluster-node-tuning-operator/tuned-vhcmx" Jul 07 11:11:18.117254 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.117202 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f5e35157-929c-493a-9da0-386b354a2acb-dbus\") pod \"global-pull-secret-syncer-vrfk8\" (UID: \"f5e35157-929c-493a-9da0-386b354a2acb\") " pod="kube-system/global-pull-secret-syncer-vrfk8" Jul 07 11:11:18.117254 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.117226 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/522a4c3d-a66a-4f5f-b8fa-e25cbf48c969-var-lib-kubelet\") pod \"tuned-vhcmx\" (UID: \"522a4c3d-a66a-4f5f-b8fa-e25cbf48c969\") " pod="openshift-cluster-node-tuning-operator/tuned-vhcmx" Jul 07 11:11:18.117254 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.117230 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/95dba470-f153-4c3c-93d6-c3fb757fec0b-host-run-netns\") pod \"multus-trhvm\" (UID: \"95dba470-f153-4c3c-93d6-c3fb757fec0b\") " pod="openshift-multus/multus-trhvm" Jul 07 11:11:18.117419 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.117266 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/95dba470-f153-4c3c-93d6-c3fb757fec0b-host-run-netns\") pod \"multus-trhvm\" (UID: \"95dba470-f153-4c3c-93d6-c3fb757fec0b\") " pod="openshift-multus/multus-trhvm" Jul 07 11:11:18.117419 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.117272 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 11:11:18.117419 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.117276 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d2340b4b-179a-4c0a-9879-13b49897c939-run-openvswitch\") pod \"ovnkube-node-tzf62\" (UID: \"d2340b4b-179a-4c0a-9879-13b49897c939\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzf62" Jul 07 11:11:18.117419 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.117309 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d2340b4b-179a-4c0a-9879-13b49897c939-run-openvswitch\") pod \"ovnkube-node-tzf62\" (UID: \"d2340b4b-179a-4c0a-9879-13b49897c939\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzf62" Jul 07 11:11:18.117419 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.117330 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/5e4c9eaf-157e-4ada-8ee8-3d9319eb26a9-socket-dir\") pod \"aws-ebs-csi-driver-node-n8tzm\" (UID: \"5e4c9eaf-157e-4ada-8ee8-3d9319eb26a9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n8tzm" Jul 07 11:11:18.117419 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.117361 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/5e4c9eaf-157e-4ada-8ee8-3d9319eb26a9-registration-dir\") pod \"aws-ebs-csi-driver-node-n8tzm\" (UID: \"5e4c9eaf-157e-4ada-8ee8-3d9319eb26a9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n8tzm" Jul 07 11:11:18.117419 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.117379 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/95dba470-f153-4c3c-93d6-c3fb757fec0b-multus-cni-dir\") pod \"multus-trhvm\" (UID: \"95dba470-f153-4c3c-93d6-c3fb757fec0b\") " pod="openshift-multus/multus-trhvm" Jul 07 11:11:18.117740 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.117422 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/95dba470-f153-4c3c-93d6-c3fb757fec0b-host-var-lib-cni-multus\") pod \"multus-trhvm\" (UID: \"95dba470-f153-4c3c-93d6-c3fb757fec0b\") " pod="openshift-multus/multus-trhvm" Jul 07 11:11:18.117740 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.117450 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/95dba470-f153-4c3c-93d6-c3fb757fec0b-multus-conf-dir\") pod \"multus-trhvm\" (UID: \"95dba470-f153-4c3c-93d6-c3fb757fec0b\") " pod="openshift-multus/multus-trhvm" Jul 07 11:11:18.117740 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.117470 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/5e4c9eaf-157e-4ada-8ee8-3d9319eb26a9-socket-dir\") pod \"aws-ebs-csi-driver-node-n8tzm\" (UID: \"5e4c9eaf-157e-4ada-8ee8-3d9319eb26a9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n8tzm" Jul 07 11:11:18.117740 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.117493 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/95dba470-f153-4c3c-93d6-c3fb757fec0b-multus-cni-dir\") pod \"multus-trhvm\" (UID: \"95dba470-f153-4c3c-93d6-c3fb757fec0b\") " pod="openshift-multus/multus-trhvm" Jul 07 11:11:18.117740 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.117498 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/95dba470-f153-4c3c-93d6-c3fb757fec0b-multus-conf-dir\") pod \"multus-trhvm\" (UID: \"95dba470-f153-4c3c-93d6-c3fb757fec0b\") " pod="openshift-multus/multus-trhvm" Jul 07 11:11:18.117740 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.117506 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/95dba470-f153-4c3c-93d6-c3fb757fec0b-host-var-lib-cni-multus\") pod \"multus-trhvm\" (UID: \"95dba470-f153-4c3c-93d6-c3fb757fec0b\") " pod="openshift-multus/multus-trhvm" Jul 07 11:11:18.117740 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.117476 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6cc36837-424c-44fd-9644-1628e5e5d144-os-release\") pod \"multus-additional-cni-plugins-6h8jc\" (UID: \"6cc36837-424c-44fd-9644-1628e5e5d144\") " pod="openshift-multus/multus-additional-cni-plugins-6h8jc" Jul 07 11:11:18.117740 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.117533 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4jfp9\" (UniqueName: \"kubernetes.io/projected/94c776b2-5ce9-4210-adf5-31efabcef2af-kube-api-access-4jfp9\") pod \"network-metrics-daemon-2q46z\" (UID: \"94c776b2-5ce9-4210-adf5-31efabcef2af\") " pod="openshift-multus/network-metrics-daemon-2q46z" Jul 07 11:11:18.117740 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.117549 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-644qj\" (UniqueName: \"kubernetes.io/projected/5e4c9eaf-157e-4ada-8ee8-3d9319eb26a9-kube-api-access-644qj\") pod \"aws-ebs-csi-driver-node-n8tzm\" (UID: \"5e4c9eaf-157e-4ada-8ee8-3d9319eb26a9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n8tzm" Jul 07 11:11:18.117740 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.117549 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6cc36837-424c-44fd-9644-1628e5e5d144-os-release\") pod \"multus-additional-cni-plugins-6h8jc\" (UID: \"6cc36837-424c-44fd-9644-1628e5e5d144\") " pod="openshift-multus/multus-additional-cni-plugins-6h8jc" Jul 07 11:11:18.117740 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.117565 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/522a4c3d-a66a-4f5f-b8fa-e25cbf48c969-etc-kubernetes\") pod \"tuned-vhcmx\" (UID: \"522a4c3d-a66a-4f5f-b8fa-e25cbf48c969\") " pod="openshift-cluster-node-tuning-operator/tuned-vhcmx" Jul 07 11:11:18.117740 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.117588 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/95dba470-f153-4c3c-93d6-c3fb757fec0b-multus-socket-dir-parent\") pod \"multus-trhvm\" (UID: \"95dba470-f153-4c3c-93d6-c3fb757fec0b\") " pod="openshift-multus/multus-trhvm" Jul 07 11:11:18.117740 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.117626 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 11:11:18.117740 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.117629 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/95dba470-f153-4c3c-93d6-c3fb757fec0b-multus-socket-dir-parent\") pod \"multus-trhvm\" (UID: \"95dba470-f153-4c3c-93d6-c3fb757fec0b\") " pod="openshift-multus/multus-trhvm" Jul 07 11:11:18.117740 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.117637 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 11:11:18.117740 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.117638 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dcf962a5f27f6d615415b27111cc76dc-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-15.ec2.internal\" (UID: \"dcf962a5f27f6d615415b27111cc76dc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-15.ec2.internal" Jul 07 11:11:18.118466 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.117684 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6cc36837-424c-44fd-9644-1628e5e5d144-cnibin\") pod \"multus-additional-cni-plugins-6h8jc\" (UID: \"6cc36837-424c-44fd-9644-1628e5e5d144\") " pod="openshift-multus/multus-additional-cni-plugins-6h8jc" Jul 07 11:11:18.118466 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.117637 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/522a4c3d-a66a-4f5f-b8fa-e25cbf48c969-etc-kubernetes\") pod \"tuned-vhcmx\" (UID: \"522a4c3d-a66a-4f5f-b8fa-e25cbf48c969\") " pod="openshift-cluster-node-tuning-operator/tuned-vhcmx" Jul 07 11:11:18.118466 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.117635 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/5e4c9eaf-157e-4ada-8ee8-3d9319eb26a9-registration-dir\") pod \"aws-ebs-csi-driver-node-n8tzm\" (UID: \"5e4c9eaf-157e-4ada-8ee8-3d9319eb26a9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n8tzm" Jul 07 11:11:18.118466 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.117714 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/6cc36837-424c-44fd-9644-1628e5e5d144-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-6h8jc\" (UID: \"6cc36837-424c-44fd-9644-1628e5e5d144\") " pod="openshift-multus/multus-additional-cni-plugins-6h8jc" Jul 07 11:11:18.118466 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.117752 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6cc36837-424c-44fd-9644-1628e5e5d144-cnibin\") pod \"multus-additional-cni-plugins-6h8jc\" (UID: \"6cc36837-424c-44fd-9644-1628e5e5d144\") " pod="openshift-multus/multus-additional-cni-plugins-6h8jc" Jul 07 11:11:18.118466 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.117757 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ztsrh\" (UniqueName: \"kubernetes.io/projected/6cc36837-424c-44fd-9644-1628e5e5d144-kube-api-access-ztsrh\") pod \"multus-additional-cni-plugins-6h8jc\" (UID: \"6cc36837-424c-44fd-9644-1628e5e5d144\") " pod="openshift-multus/multus-additional-cni-plugins-6h8jc" Jul 07 11:11:18.118466 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.117781 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dcf962a5f27f6d615415b27111cc76dc-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-15.ec2.internal\" (UID: \"dcf962a5f27f6d615415b27111cc76dc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-15.ec2.internal" Jul 07 11:11:18.118466 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.117791 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/e93d8868-ca25-4c0a-9f25-da7b8c286f60-serviceca\") pod \"node-ca-528n2\" (UID: \"e93d8868-ca25-4c0a-9f25-da7b8c286f60\") " pod="openshift-image-registry/node-ca-528n2" Jul 07 11:11:18.118466 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.117817 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 11:11:18.118466 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.117817 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/5e4c9eaf-157e-4ada-8ee8-3d9319eb26a9-device-dir\") pod \"aws-ebs-csi-driver-node-n8tzm\" (UID: \"5e4c9eaf-157e-4ada-8ee8-3d9319eb26a9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n8tzm" Jul 07 11:11:18.118466 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.117842 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/522a4c3d-a66a-4f5f-b8fa-e25cbf48c969-etc-sysctl-conf\") pod \"tuned-vhcmx\" (UID: \"522a4c3d-a66a-4f5f-b8fa-e25cbf48c969\") " pod="openshift-cluster-node-tuning-operator/tuned-vhcmx" Jul 07 11:11:18.118466 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.117868 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d2340b4b-179a-4c0a-9879-13b49897c939-host-run-ovn-kubernetes\") pod \"ovnkube-node-tzf62\" (UID: \"d2340b4b-179a-4c0a-9879-13b49897c939\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzf62" Jul 07 11:11:18.118466 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.117891 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tpn9m\" (UniqueName: \"kubernetes.io/projected/d2340b4b-179a-4c0a-9879-13b49897c939-kube-api-access-tpn9m\") pod \"ovnkube-node-tzf62\" (UID: \"d2340b4b-179a-4c0a-9879-13b49897c939\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzf62" Jul 07 11:11:18.118466 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.117914 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a0c235cfc97783f979ae1f6fa1e22e58-config\") pod \"kube-apiserver-proxy-ip-10-0-136-15.ec2.internal\" (UID: \"a0c235cfc97783f979ae1f6fa1e22e58\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-15.ec2.internal" Jul 07 11:11:18.119177 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.117956 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/5e4c9eaf-157e-4ada-8ee8-3d9319eb26a9-sys-fs\") pod \"aws-ebs-csi-driver-node-n8tzm\" (UID: \"5e4c9eaf-157e-4ada-8ee8-3d9319eb26a9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n8tzm" Jul 07 11:11:18.119177 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.117980 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/522a4c3d-a66a-4f5f-b8fa-e25cbf48c969-run\") pod \"tuned-vhcmx\" (UID: \"522a4c3d-a66a-4f5f-b8fa-e25cbf48c969\") " pod="openshift-cluster-node-tuning-operator/tuned-vhcmx" Jul 07 11:11:18.119177 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.118002 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/95dba470-f153-4c3c-93d6-c3fb757fec0b-hostroot\") pod \"multus-trhvm\" (UID: \"95dba470-f153-4c3c-93d6-c3fb757fec0b\") " pod="openshift-multus/multus-trhvm" Jul 07 11:11:18.119177 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.118053 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 11:11:18.119177 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.118067 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/5e4c9eaf-157e-4ada-8ee8-3d9319eb26a9-device-dir\") pod \"aws-ebs-csi-driver-node-n8tzm\" (UID: \"5e4c9eaf-157e-4ada-8ee8-3d9319eb26a9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n8tzm" Jul 07 11:11:18.119177 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.118080 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/522a4c3d-a66a-4f5f-b8fa-e25cbf48c969-etc-sysctl-conf\") pod \"tuned-vhcmx\" (UID: \"522a4c3d-a66a-4f5f-b8fa-e25cbf48c969\") " pod="openshift-cluster-node-tuning-operator/tuned-vhcmx" Jul 07 11:11:18.119177 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.118085 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d2340b4b-179a-4c0a-9879-13b49897c939-run-ovn\") pod \"ovnkube-node-tzf62\" (UID: \"d2340b4b-179a-4c0a-9879-13b49897c939\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzf62" Jul 07 11:11:18.119177 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.118107 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d2340b4b-179a-4c0a-9879-13b49897c939-host-run-ovn-kubernetes\") pod \"ovnkube-node-tzf62\" (UID: \"d2340b4b-179a-4c0a-9879-13b49897c939\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzf62" Jul 07 11:11:18.119177 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.118134 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d2340b4b-179a-4c0a-9879-13b49897c939-host-cni-bin\") pod \"ovnkube-node-tzf62\" (UID: \"d2340b4b-179a-4c0a-9879-13b49897c939\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzf62" Jul 07 11:11:18.119177 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.118161 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/522a4c3d-a66a-4f5f-b8fa-e25cbf48c969-etc-modprobe-d\") pod \"tuned-vhcmx\" (UID: \"522a4c3d-a66a-4f5f-b8fa-e25cbf48c969\") " pod="openshift-cluster-node-tuning-operator/tuned-vhcmx" Jul 07 11:11:18.119177 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.118189 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/522a4c3d-a66a-4f5f-b8fa-e25cbf48c969-run\") pod \"tuned-vhcmx\" (UID: \"522a4c3d-a66a-4f5f-b8fa-e25cbf48c969\") " pod="openshift-cluster-node-tuning-operator/tuned-vhcmx" Jul 07 11:11:18.119177 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.118187 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/522a4c3d-a66a-4f5f-b8fa-e25cbf48c969-lib-modules\") pod \"tuned-vhcmx\" (UID: \"522a4c3d-a66a-4f5f-b8fa-e25cbf48c969\") " pod="openshift-cluster-node-tuning-operator/tuned-vhcmx" Jul 07 11:11:18.119177 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.118235 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a0c235cfc97783f979ae1f6fa1e22e58-config\") pod \"kube-apiserver-proxy-ip-10-0-136-15.ec2.internal\" (UID: \"a0c235cfc97783f979ae1f6fa1e22e58\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-15.ec2.internal" Jul 07 11:11:18.119177 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.118276 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d2340b4b-179a-4c0a-9879-13b49897c939-host-cni-bin\") pod \"ovnkube-node-tzf62\" (UID: \"d2340b4b-179a-4c0a-9879-13b49897c939\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzf62" Jul 07 11:11:18.119177 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.118284 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d2340b4b-179a-4c0a-9879-13b49897c939-host-kubelet\") pod \"ovnkube-node-tzf62\" (UID: \"d2340b4b-179a-4c0a-9879-13b49897c939\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzf62" Jul 07 11:11:18.119904 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.118340 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/95dba470-f153-4c3c-93d6-c3fb757fec0b-hostroot\") pod \"multus-trhvm\" (UID: \"95dba470-f153-4c3c-93d6-c3fb757fec0b\") " pod="openshift-multus/multus-trhvm" Jul 07 11:11:18.119904 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.118344 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasSufficientMemory" Jul 07 11:11:18.119904 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.118363 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/522a4c3d-a66a-4f5f-b8fa-e25cbf48c969-lib-modules\") pod \"tuned-vhcmx\" (UID: \"522a4c3d-a66a-4f5f-b8fa-e25cbf48c969\") " pod="openshift-cluster-node-tuning-operator/tuned-vhcmx" Jul 07 11:11:18.119904 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.118373 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 11:11:18.119904 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.118390 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/5e4c9eaf-157e-4ada-8ee8-3d9319eb26a9-sys-fs\") pod \"aws-ebs-csi-driver-node-n8tzm\" (UID: \"5e4c9eaf-157e-4ada-8ee8-3d9319eb26a9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n8tzm" Jul 07 11:11:18.119904 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.118418 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasSufficientPID" Jul 07 11:11:18.119904 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.118423 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d2340b4b-179a-4c0a-9879-13b49897c939-ovn-node-metrics-cert\") pod \"ovnkube-node-tzf62\" (UID: \"d2340b4b-179a-4c0a-9879-13b49897c939\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzf62" Jul 07 11:11:18.119904 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.118451 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/522a4c3d-a66a-4f5f-b8fa-e25cbf48c969-etc-tuned\") pod \"tuned-vhcmx\" (UID: \"522a4c3d-a66a-4f5f-b8fa-e25cbf48c969\") " pod="openshift-cluster-node-tuning-operator/tuned-vhcmx" Jul 07 11:11:18.119904 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.118484 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 11:11:18.119904 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.118490 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasSufficientMemory" Jul 07 11:11:18.119904 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.118495 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/6cc36837-424c-44fd-9644-1628e5e5d144-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-6h8jc\" (UID: \"6cc36837-424c-44fd-9644-1628e5e5d144\") " pod="openshift-multus/multus-additional-cni-plugins-6h8jc" Jul 07 11:11:18.119904 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.118513 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 11:11:18.119904 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.118529 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasSufficientPID" Jul 07 11:11:18.119904 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.118544 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/522a4c3d-a66a-4f5f-b8fa-e25cbf48c969-etc-modprobe-d\") pod \"tuned-vhcmx\" (UID: \"522a4c3d-a66a-4f5f-b8fa-e25cbf48c969\") " pod="openshift-cluster-node-tuning-operator/tuned-vhcmx" Jul 07 11:11:18.119904 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.118460 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/e93d8868-ca25-4c0a-9f25-da7b8c286f60-serviceca\") pod \"node-ca-528n2\" (UID: \"e93d8868-ca25-4c0a-9f25-da7b8c286f60\") " pod="openshift-image-registry/node-ca-528n2" Jul 07 11:11:18.119904 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.118604 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasSufficientMemory" Jul 07 11:11:18.119904 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.118627 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 11:11:18.119904 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.118626 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d2340b4b-179a-4c0a-9879-13b49897c939-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-tzf62\" (UID: \"d2340b4b-179a-4c0a-9879-13b49897c939\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzf62" Jul 07 11:11:18.119904 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.118630 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d2340b4b-179a-4c0a-9879-13b49897c939-host-kubelet\") pod \"ovnkube-node-tzf62\" (UID: \"d2340b4b-179a-4c0a-9879-13b49897c939\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzf62" Jul 07 11:11:18.119904 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.118642 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasSufficientPID" Jul 07 11:11:18.120986 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.118651 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d2340b4b-179a-4c0a-9879-13b49897c939-ovnkube-script-lib\") pod \"ovnkube-node-tzf62\" (UID: \"d2340b4b-179a-4c0a-9879-13b49897c939\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzf62" Jul 07 11:11:18.120986 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.118786 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d2340b4b-179a-4c0a-9879-13b49897c939-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-tzf62\" (UID: \"d2340b4b-179a-4c0a-9879-13b49897c939\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzf62" Jul 07 11:11:18.120986 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.118813 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/dcf962a5f27f6d615415b27111cc76dc-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-15.ec2.internal\" (UID: \"dcf962a5f27f6d615415b27111cc76dc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-15.ec2.internal" Jul 07 11:11:18.120986 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.118870 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e93d8868-ca25-4c0a-9f25-da7b8c286f60-host\") pod \"node-ca-528n2\" (UID: \"e93d8868-ca25-4c0a-9f25-da7b8c286f60\") " pod="openshift-image-registry/node-ca-528n2" Jul 07 11:11:18.120986 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.118904 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/522a4c3d-a66a-4f5f-b8fa-e25cbf48c969-etc-systemd\") pod \"tuned-vhcmx\" (UID: \"522a4c3d-a66a-4f5f-b8fa-e25cbf48c969\") " pod="openshift-cluster-node-tuning-operator/tuned-vhcmx" Jul 07 11:11:18.120986 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.119124 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e93d8868-ca25-4c0a-9f25-da7b8c286f60-host\") pod \"node-ca-528n2\" (UID: \"e93d8868-ca25-4c0a-9f25-da7b8c286f60\") " pod="openshift-image-registry/node-ca-528n2" Jul 07 11:11:18.120986 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.119137 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/dcf962a5f27f6d615415b27111cc76dc-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-15.ec2.internal\" (UID: \"dcf962a5f27f6d615415b27111cc76dc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-15.ec2.internal" Jul 07 11:11:18.120986 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.119170 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/522a4c3d-a66a-4f5f-b8fa-e25cbf48c969-sys\") pod \"tuned-vhcmx\" (UID: \"522a4c3d-a66a-4f5f-b8fa-e25cbf48c969\") " pod="openshift-cluster-node-tuning-operator/tuned-vhcmx" Jul 07 11:11:18.120986 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.119188 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d2340b4b-179a-4c0a-9879-13b49897c939-run-ovn\") pod \"ovnkube-node-tzf62\" (UID: \"d2340b4b-179a-4c0a-9879-13b49897c939\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzf62" Jul 07 11:11:18.120986 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.119221 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 11:11:18.120986 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.119235 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/522a4c3d-a66a-4f5f-b8fa-e25cbf48c969-etc-systemd\") pod \"tuned-vhcmx\" (UID: \"522a4c3d-a66a-4f5f-b8fa-e25cbf48c969\") " pod="openshift-cluster-node-tuning-operator/tuned-vhcmx" Jul 07 11:11:18.120986 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.119280 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/522a4c3d-a66a-4f5f-b8fa-e25cbf48c969-sys\") pod \"tuned-vhcmx\" (UID: \"522a4c3d-a66a-4f5f-b8fa-e25cbf48c969\") " pod="openshift-cluster-node-tuning-operator/tuned-vhcmx" Jul 07 11:11:18.120986 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.119309 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/a9da5840-e6b4-4719-98cd-ef0d46d8470c-hosts-file\") pod \"node-resolver-77v22\" (UID: \"a9da5840-e6b4-4719-98cd-ef0d46d8470c\") " pod="openshift-dns/node-resolver-77v22" Jul 07 11:11:18.120986 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.119354 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lw46c\" (UniqueName: \"kubernetes.io/projected/a9da5840-e6b4-4719-98cd-ef0d46d8470c-kube-api-access-lw46c\") pod \"node-resolver-77v22\" (UID: \"a9da5840-e6b4-4719-98cd-ef0d46d8470c\") " pod="openshift-dns/node-resolver-77v22" Jul 07 11:11:18.120986 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.119390 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/95dba470-f153-4c3c-93d6-c3fb757fec0b-etc-kubernetes\") pod \"multus-trhvm\" (UID: \"95dba470-f153-4c3c-93d6-c3fb757fec0b\") " pod="openshift-multus/multus-trhvm" Jul 07 11:11:18.121883 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.119431 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d2340b4b-179a-4c0a-9879-13b49897c939-ovnkube-config\") pod \"ovnkube-node-tzf62\" (UID: \"d2340b4b-179a-4c0a-9879-13b49897c939\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzf62" Jul 07 11:11:18.121883 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.119461 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasSufficientMemory" Jul 07 11:11:18.121883 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.119468 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/95dba470-f153-4c3c-93d6-c3fb757fec0b-etc-kubernetes\") pod \"multus-trhvm\" (UID: \"95dba470-f153-4c3c-93d6-c3fb757fec0b\") " pod="openshift-multus/multus-trhvm" Jul 07 11:11:18.121883 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.119462 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d2340b4b-179a-4c0a-9879-13b49897c939-ovnkube-script-lib\") pod \"ovnkube-node-tzf62\" (UID: \"d2340b4b-179a-4c0a-9879-13b49897c939\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzf62" Jul 07 11:11:18.121883 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.119497 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasSufficientMemory" Jul 07 11:11:18.121883 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.119532 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 11:11:18.121883 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.119545 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasSufficientPID" Jul 07 11:11:18.121883 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.119555 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 11:11:18.121883 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.119489 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 11:11:18.121883 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.119495 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kfm4r\" (UniqueName: \"kubernetes.io/projected/e93d8868-ca25-4c0a-9f25-da7b8c286f60-kube-api-access-kfm4r\") pod \"node-ca-528n2\" (UID: \"e93d8868-ca25-4c0a-9f25-da7b8c286f60\") " pod="openshift-image-registry/node-ca-528n2" Jul 07 11:11:18.121883 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.119591 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasSufficientPID" Jul 07 11:11:18.121883 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.119605 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d4ebce69-4659-439d-b4fa-8e9f4b17ec2a-host-slash\") pod \"iptables-alerter-dvhgx\" (UID: \"d4ebce69-4659-439d-b4fa-8e9f4b17ec2a\") " pod="openshift-network-operator/iptables-alerter-dvhgx" Jul 07 11:11:18.121883 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.119629 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f5e35157-929c-493a-9da0-386b354a2acb-kubelet-config\") pod \"global-pull-secret-syncer-vrfk8\" (UID: \"f5e35157-929c-493a-9da0-386b354a2acb\") " pod="kube-system/global-pull-secret-syncer-vrfk8" Jul 07 11:11:18.121883 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.119669 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/95dba470-f153-4c3c-93d6-c3fb757fec0b-system-cni-dir\") pod \"multus-trhvm\" (UID: \"95dba470-f153-4c3c-93d6-c3fb757fec0b\") " pod="openshift-multus/multus-trhvm" Jul 07 11:11:18.121883 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.119700 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/95dba470-f153-4c3c-93d6-c3fb757fec0b-cnibin\") pod \"multus-trhvm\" (UID: \"95dba470-f153-4c3c-93d6-c3fb757fec0b\") " pod="openshift-multus/multus-trhvm" Jul 07 11:11:18.121883 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.119741 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d2340b4b-179a-4c0a-9879-13b49897c939-etc-openvswitch\") pod \"ovnkube-node-tzf62\" (UID: \"d2340b4b-179a-4c0a-9879-13b49897c939\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzf62" Jul 07 11:11:18.121883 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.119767 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d2340b4b-179a-4c0a-9879-13b49897c939-node-log\") pod \"ovnkube-node-tzf62\" (UID: \"d2340b4b-179a-4c0a-9879-13b49897c939\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzf62" Jul 07 11:11:18.121883 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.119790 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d2340b4b-179a-4c0a-9879-13b49897c939-systemd-units\") pod \"ovnkube-node-tzf62\" (UID: \"d2340b4b-179a-4c0a-9879-13b49897c939\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzf62" Jul 07 11:11:18.122772 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.119818 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gg4ts\" (UniqueName: \"kubernetes.io/projected/d4ebce69-4659-439d-b4fa-8e9f4b17ec2a-kube-api-access-gg4ts\") pod \"iptables-alerter-dvhgx\" (UID: \"d4ebce69-4659-439d-b4fa-8e9f4b17ec2a\") " pod="openshift-network-operator/iptables-alerter-dvhgx" Jul 07 11:11:18.122772 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.119833 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/95dba470-f153-4c3c-93d6-c3fb757fec0b-system-cni-dir\") pod \"multus-trhvm\" (UID: \"95dba470-f153-4c3c-93d6-c3fb757fec0b\") " pod="openshift-multus/multus-trhvm" Jul 07 11:11:18.122772 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.119844 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/95dba470-f153-4c3c-93d6-c3fb757fec0b-host-run-k8s-cni-cncf-io\") pod \"multus-trhvm\" (UID: \"95dba470-f153-4c3c-93d6-c3fb757fec0b\") " pod="openshift-multus/multus-trhvm" Jul 07 11:11:18.122772 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.119875 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d2340b4b-179a-4c0a-9879-13b49897c939-node-log\") pod \"ovnkube-node-tzf62\" (UID: \"d2340b4b-179a-4c0a-9879-13b49897c939\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzf62" Jul 07 11:11:18.122772 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.119882 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/522a4c3d-a66a-4f5f-b8fa-e25cbf48c969-etc-sysctl-d\") pod \"tuned-vhcmx\" (UID: \"522a4c3d-a66a-4f5f-b8fa-e25cbf48c969\") " pod="openshift-cluster-node-tuning-operator/tuned-vhcmx" Jul 07 11:11:18.122772 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.119886 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/95dba470-f153-4c3c-93d6-c3fb757fec0b-host-run-k8s-cni-cncf-io\") pod \"multus-trhvm\" (UID: \"95dba470-f153-4c3c-93d6-c3fb757fec0b\") " pod="openshift-multus/multus-trhvm" Jul 07 11:11:18.122772 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.119918 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/522a4c3d-a66a-4f5f-b8fa-e25cbf48c969-host\") pod \"tuned-vhcmx\" (UID: \"522a4c3d-a66a-4f5f-b8fa-e25cbf48c969\") " pod="openshift-cluster-node-tuning-operator/tuned-vhcmx" Jul 07 11:11:18.122772 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.119947 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d2340b4b-179a-4c0a-9879-13b49897c939-log-socket\") pod \"ovnkube-node-tzf62\" (UID: \"d2340b4b-179a-4c0a-9879-13b49897c939\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzf62" Jul 07 11:11:18.122772 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.119974 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d2340b4b-179a-4c0a-9879-13b49897c939-etc-openvswitch\") pod \"ovnkube-node-tzf62\" (UID: \"d2340b4b-179a-4c0a-9879-13b49897c939\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzf62" Jul 07 11:11:18.122772 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.119994 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/663d4847-eafe-43d6-a4b8-2bec15919119-konnectivity-ca\") pod \"konnectivity-agent-4b5qw\" (UID: \"663d4847-eafe-43d6-a4b8-2bec15919119\") " pod="kube-system/konnectivity-agent-4b5qw" Jul 07 11:11:18.122772 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.119962 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d2340b4b-179a-4c0a-9879-13b49897c939-ovnkube-config\") pod \"ovnkube-node-tzf62\" (UID: \"d2340b4b-179a-4c0a-9879-13b49897c939\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzf62" Jul 07 11:11:18.122772 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.120028 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/522a4c3d-a66a-4f5f-b8fa-e25cbf48c969-etc-sysctl-d\") pod \"tuned-vhcmx\" (UID: \"522a4c3d-a66a-4f5f-b8fa-e25cbf48c969\") " pod="openshift-cluster-node-tuning-operator/tuned-vhcmx" Jul 07 11:11:18.122772 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.120032 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7nvg6\" (UniqueName: \"kubernetes.io/projected/b9ca66a2-4c5a-4816-a081-bb0fecbdab5d-kube-api-access-7nvg6\") pod \"network-check-target-sc5bl\" (UID: \"b9ca66a2-4c5a-4816-a081-bb0fecbdab5d\") " pod="openshift-network-diagnostics/network-check-target-sc5bl" Jul 07 11:11:18.122772 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.119958 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d2340b4b-179a-4c0a-9879-13b49897c939-systemd-units\") pod \"ovnkube-node-tzf62\" (UID: \"d2340b4b-179a-4c0a-9879-13b49897c939\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzf62" Jul 07 11:11:18.122772 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.120031 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/95dba470-f153-4c3c-93d6-c3fb757fec0b-cnibin\") pod \"multus-trhvm\" (UID: \"95dba470-f153-4c3c-93d6-c3fb757fec0b\") " pod="openshift-multus/multus-trhvm" Jul 07 11:11:18.123616 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.119919 2562 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 07 11:11:18.123616 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.120039 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/522a4c3d-a66a-4f5f-b8fa-e25cbf48c969-host\") pod \"tuned-vhcmx\" (UID: \"522a4c3d-a66a-4f5f-b8fa-e25cbf48c969\") " pod="openshift-cluster-node-tuning-operator/tuned-vhcmx" Jul 07 11:11:18.123616 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.120063 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d4ebce69-4659-439d-b4fa-8e9f4b17ec2a-iptables-alerter-script\") pod \"iptables-alerter-dvhgx\" (UID: \"d4ebce69-4659-439d-b4fa-8e9f4b17ec2a\") " pod="openshift-network-operator/iptables-alerter-dvhgx" Jul 07 11:11:18.123616 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.120076 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d2340b4b-179a-4c0a-9879-13b49897c939-log-socket\") pod \"ovnkube-node-tzf62\" (UID: \"d2340b4b-179a-4c0a-9879-13b49897c939\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzf62" Jul 07 11:11:18.123616 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.120117 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasSufficientMemory" Jul 07 11:11:18.123616 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.120150 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 11:11:18.123616 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.120164 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasSufficientPID" Jul 07 11:11:18.123616 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.120204 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/95dba470-f153-4c3c-93d6-c3fb757fec0b-os-release\") pod \"multus-trhvm\" (UID: \"95dba470-f153-4c3c-93d6-c3fb757fec0b\") " pod="openshift-multus/multus-trhvm" Jul 07 11:11:18.123616 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.120277 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasSufficientMemory" Jul 07 11:11:18.123616 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.120299 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 11:11:18.123616 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.120318 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasSufficientPID" Jul 07 11:11:18.123616 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.120358 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6rsn9\" (UniqueName: \"kubernetes.io/projected/95dba470-f153-4c3c-93d6-c3fb757fec0b-kube-api-access-6rsn9\") pod \"multus-trhvm\" (UID: \"95dba470-f153-4c3c-93d6-c3fb757fec0b\") " pod="openshift-multus/multus-trhvm" Jul 07 11:11:18.123616 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.120454 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasSufficientMemory" Jul 07 11:11:18.123616 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.120475 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 11:11:18.123616 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.120487 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasSufficientPID" Jul 07 11:11:18.123616 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.120537 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d2340b4b-179a-4c0a-9879-13b49897c939-host-slash\") pod \"ovnkube-node-tzf62\" (UID: \"d2340b4b-179a-4c0a-9879-13b49897c939\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzf62" Jul 07 11:11:18.123616 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.120568 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d2340b4b-179a-4c0a-9879-13b49897c939-env-overrides\") pod \"ovnkube-node-tzf62\" (UID: \"d2340b4b-179a-4c0a-9879-13b49897c939\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzf62" Jul 07 11:11:18.123616 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.120589 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/663d4847-eafe-43d6-a4b8-2bec15919119-konnectivity-ca\") pod \"konnectivity-agent-4b5qw\" (UID: \"663d4847-eafe-43d6-a4b8-2bec15919119\") " pod="kube-system/konnectivity-agent-4b5qw" Jul 07 11:11:18.123616 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.120611 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/95dba470-f153-4c3c-93d6-c3fb757fec0b-os-release\") pod \"multus-trhvm\" (UID: \"95dba470-f153-4c3c-93d6-c3fb757fec0b\") " pod="openshift-multus/multus-trhvm" Jul 07 11:11:18.123616 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:18.120600 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-15.ec2.internal\" not found" Jul 07 11:11:18.123616 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.120709 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 11:11:18.124517 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.120712 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d2340b4b-179a-4c0a-9879-13b49897c939-host-slash\") pod \"ovnkube-node-tzf62\" (UID: \"d2340b4b-179a-4c0a-9879-13b49897c939\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzf62" Jul 07 11:11:18.124517 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.120712 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6cc36837-424c-44fd-9644-1628e5e5d144-system-cni-dir\") pod \"multus-additional-cni-plugins-6h8jc\" (UID: \"6cc36837-424c-44fd-9644-1628e5e5d144\") " pod="openshift-multus/multus-additional-cni-plugins-6h8jc" Jul 07 11:11:18.124517 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.120767 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6cc36837-424c-44fd-9644-1628e5e5d144-system-cni-dir\") pod \"multus-additional-cni-plugins-6h8jc\" (UID: \"6cc36837-424c-44fd-9644-1628e5e5d144\") " pod="openshift-multus/multus-additional-cni-plugins-6h8jc" Jul 07 11:11:18.124517 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.120810 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6cc36837-424c-44fd-9644-1628e5e5d144-cni-binary-copy\") pod \"multus-additional-cni-plugins-6h8jc\" (UID: \"6cc36837-424c-44fd-9644-1628e5e5d144\") " pod="openshift-multus/multus-additional-cni-plugins-6h8jc" Jul 07 11:11:18.124517 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.120940 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/6cc36837-424c-44fd-9644-1628e5e5d144-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-6h8jc\" (UID: \"6cc36837-424c-44fd-9644-1628e5e5d144\") " pod="openshift-multus/multus-additional-cni-plugins-6h8jc" Jul 07 11:11:18.124517 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.121031 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/94c776b2-5ce9-4210-adf5-31efabcef2af-metrics-certs\") pod \"network-metrics-daemon-2q46z\" (UID: \"94c776b2-5ce9-4210-adf5-31efabcef2af\") " pod="openshift-multus/network-metrics-daemon-2q46z" Jul 07 11:11:18.124517 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.121062 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/95dba470-f153-4c3c-93d6-c3fb757fec0b-cni-binary-copy\") pod \"multus-trhvm\" (UID: \"95dba470-f153-4c3c-93d6-c3fb757fec0b\") " pod="openshift-multus/multus-trhvm" Jul 07 11:11:18.124517 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.121105 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/95dba470-f153-4c3c-93d6-c3fb757fec0b-host-var-lib-kubelet\") pod \"multus-trhvm\" (UID: \"95dba470-f153-4c3c-93d6-c3fb757fec0b\") " pod="openshift-multus/multus-trhvm" Jul 07 11:11:18.124517 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.121131 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/95dba470-f153-4c3c-93d6-c3fb757fec0b-host-run-multus-certs\") pod \"multus-trhvm\" (UID: \"95dba470-f153-4c3c-93d6-c3fb757fec0b\") " pod="openshift-multus/multus-trhvm" Jul 07 11:11:18.124517 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.121157 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d2340b4b-179a-4c0a-9879-13b49897c939-host-run-netns\") pod \"ovnkube-node-tzf62\" (UID: \"d2340b4b-179a-4c0a-9879-13b49897c939\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzf62" Jul 07 11:11:18.124517 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.121206 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d2340b4b-179a-4c0a-9879-13b49897c939-var-lib-openvswitch\") pod \"ovnkube-node-tzf62\" (UID: \"d2340b4b-179a-4c0a-9879-13b49897c939\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzf62" Jul 07 11:11:18.124517 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.121225 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 11:11:18.124517 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.121233 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/5e4c9eaf-157e-4ada-8ee8-3d9319eb26a9-kubelet-dir\") pod \"aws-ebs-csi-driver-node-n8tzm\" (UID: \"5e4c9eaf-157e-4ada-8ee8-3d9319eb26a9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n8tzm" Jul 07 11:11:18.124517 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.121234 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d2340b4b-179a-4c0a-9879-13b49897c939-env-overrides\") pod \"ovnkube-node-tzf62\" (UID: \"d2340b4b-179a-4c0a-9879-13b49897c939\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzf62" Jul 07 11:11:18.124517 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.121256 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/522a4c3d-a66a-4f5f-b8fa-e25cbf48c969-tmp\") pod \"tuned-vhcmx\" (UID: \"522a4c3d-a66a-4f5f-b8fa-e25cbf48c969\") " pod="openshift-cluster-node-tuning-operator/tuned-vhcmx" Jul 07 11:11:18.124976 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.121281 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f5e35157-929c-493a-9da0-386b354a2acb-original-pull-secret\") pod \"global-pull-secret-syncer-vrfk8\" (UID: \"f5e35157-929c-493a-9da0-386b354a2acb\") " pod="kube-system/global-pull-secret-syncer-vrfk8" Jul 07 11:11:18.124976 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.121325 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 11:11:18.124976 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.121324 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d2340b4b-179a-4c0a-9879-13b49897c939-host-run-netns\") pod \"ovnkube-node-tzf62\" (UID: \"d2340b4b-179a-4c0a-9879-13b49897c939\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzf62" Jul 07 11:11:18.124976 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.121368 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/95dba470-f153-4c3c-93d6-c3fb757fec0b-host-run-multus-certs\") pod \"multus-trhvm\" (UID: \"95dba470-f153-4c3c-93d6-c3fb757fec0b\") " pod="openshift-multus/multus-trhvm" Jul 07 11:11:18.124976 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.121398 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d2340b4b-179a-4c0a-9879-13b49897c939-var-lib-openvswitch\") pod \"ovnkube-node-tzf62\" (UID: \"d2340b4b-179a-4c0a-9879-13b49897c939\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzf62" Jul 07 11:11:18.124976 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.121418 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6cc36837-424c-44fd-9644-1628e5e5d144-cni-binary-copy\") pod \"multus-additional-cni-plugins-6h8jc\" (UID: \"6cc36837-424c-44fd-9644-1628e5e5d144\") " pod="openshift-multus/multus-additional-cni-plugins-6h8jc" Jul 07 11:11:18.124976 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.121420 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/95dba470-f153-4c3c-93d6-c3fb757fec0b-host-var-lib-kubelet\") pod \"multus-trhvm\" (UID: \"95dba470-f153-4c3c-93d6-c3fb757fec0b\") " pod="openshift-multus/multus-trhvm" Jul 07 11:11:18.124976 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.121439 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6cc36837-424c-44fd-9644-1628e5e5d144-tuning-conf-dir\") pod \"multus-additional-cni-plugins-6h8jc\" (UID: \"6cc36837-424c-44fd-9644-1628e5e5d144\") " pod="openshift-multus/multus-additional-cni-plugins-6h8jc" Jul 07 11:11:18.124976 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.121467 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/663d4847-eafe-43d6-a4b8-2bec15919119-agent-certs\") pod \"konnectivity-agent-4b5qw\" (UID: \"663d4847-eafe-43d6-a4b8-2bec15919119\") " pod="kube-system/konnectivity-agent-4b5qw" Jul 07 11:11:18.124976 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.121485 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/5e4c9eaf-157e-4ada-8ee8-3d9319eb26a9-kubelet-dir\") pod \"aws-ebs-csi-driver-node-n8tzm\" (UID: \"5e4c9eaf-157e-4ada-8ee8-3d9319eb26a9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n8tzm" Jul 07 11:11:18.124976 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.121533 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/522a4c3d-a66a-4f5f-b8fa-e25cbf48c969-etc-sysconfig\") pod \"tuned-vhcmx\" (UID: \"522a4c3d-a66a-4f5f-b8fa-e25cbf48c969\") " pod="openshift-cluster-node-tuning-operator/tuned-vhcmx" Jul 07 11:11:18.124976 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.121559 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a9da5840-e6b4-4719-98cd-ef0d46d8470c-tmp-dir\") pod \"node-resolver-77v22\" (UID: \"a9da5840-e6b4-4719-98cd-ef0d46d8470c\") " pod="openshift-dns/node-resolver-77v22" Jul 07 11:11:18.124976 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.121583 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/95dba470-f153-4c3c-93d6-c3fb757fec0b-host-var-lib-cni-bin\") pod \"multus-trhvm\" (UID: \"95dba470-f153-4c3c-93d6-c3fb757fec0b\") " pod="openshift-multus/multus-trhvm" Jul 07 11:11:18.124976 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.121621 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/95dba470-f153-4c3c-93d6-c3fb757fec0b-multus-daemon-config\") pod \"multus-trhvm\" (UID: \"95dba470-f153-4c3c-93d6-c3fb757fec0b\") " pod="openshift-multus/multus-trhvm" Jul 07 11:11:18.124976 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.121662 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d2340b4b-179a-4c0a-9879-13b49897c939-run-systemd\") pod \"ovnkube-node-tzf62\" (UID: \"d2340b4b-179a-4c0a-9879-13b49897c939\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzf62" Jul 07 11:11:18.125422 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.121742 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/6cc36837-424c-44fd-9644-1628e5e5d144-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-6h8jc\" (UID: \"6cc36837-424c-44fd-9644-1628e5e5d144\") " pod="openshift-multus/multus-additional-cni-plugins-6h8jc" Jul 07 11:11:18.125422 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.121779 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d2340b4b-179a-4c0a-9879-13b49897c939-run-systemd\") pod \"ovnkube-node-tzf62\" (UID: \"d2340b4b-179a-4c0a-9879-13b49897c939\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzf62" Jul 07 11:11:18.125422 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.121854 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/522a4c3d-a66a-4f5f-b8fa-e25cbf48c969-etc-sysconfig\") pod \"tuned-vhcmx\" (UID: \"522a4c3d-a66a-4f5f-b8fa-e25cbf48c969\") " pod="openshift-cluster-node-tuning-operator/tuned-vhcmx" Jul 07 11:11:18.125422 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.121866 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 11:11:18.125422 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.121899 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasSufficientMemory" Jul 07 11:11:18.125422 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.121920 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/95dba470-f153-4c3c-93d6-c3fb757fec0b-host-var-lib-cni-bin\") pod \"multus-trhvm\" (UID: \"95dba470-f153-4c3c-93d6-c3fb757fec0b\") " pod="openshift-multus/multus-trhvm" Jul 07 11:11:18.125422 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.121947 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 11:11:18.125422 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.121963 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasSufficientPID" Jul 07 11:11:18.125422 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.122055 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasSufficientMemory" Jul 07 11:11:18.125422 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.122079 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 11:11:18.125422 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.122094 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasSufficientPID" Jul 07 11:11:18.125422 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.122248 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasSufficientMemory" Jul 07 11:11:18.125422 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.122269 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 11:11:18.125422 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.122283 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasSufficientPID" Jul 07 11:11:18.125422 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.122382 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/95dba470-f153-4c3c-93d6-c3fb757fec0b-multus-daemon-config\") pod \"multus-trhvm\" (UID: \"95dba470-f153-4c3c-93d6-c3fb757fec0b\") " pod="openshift-multus/multus-trhvm" Jul 07 11:11:18.125422 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.122727 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/95dba470-f153-4c3c-93d6-c3fb757fec0b-cni-binary-copy\") pod \"multus-trhvm\" (UID: \"95dba470-f153-4c3c-93d6-c3fb757fec0b\") " pod="openshift-multus/multus-trhvm" Jul 07 11:11:18.125422 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.122829 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6cc36837-424c-44fd-9644-1628e5e5d144-tuning-conf-dir\") pod \"multus-additional-cni-plugins-6h8jc\" (UID: \"6cc36837-424c-44fd-9644-1628e5e5d144\") " pod="openshift-multus/multus-additional-cni-plugins-6h8jc" Jul 07 11:11:18.125422 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.122907 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasSufficientMemory" Jul 07 11:11:18.125422 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.122933 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 11:11:18.125422 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.122950 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasSufficientPID" Jul 07 11:11:18.125422 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:18.123015 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 11:11:18.126028 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:18.123159 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/94c776b2-5ce9-4210-adf5-31efabcef2af-metrics-certs podName:94c776b2-5ce9-4210-adf5-31efabcef2af nodeName:}" failed. No retries permitted until 2026-07-07 11:11:18.623142904 +0000 UTC m=+3.037012079 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/94c776b2-5ce9-4210-adf5-31efabcef2af-metrics-certs") pod "network-metrics-daemon-2q46z" (UID: "94c776b2-5ce9-4210-adf5-31efabcef2af") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 11:11:18.126028 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.124037 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/522a4c3d-a66a-4f5f-b8fa-e25cbf48c969-etc-tuned\") pod \"tuned-vhcmx\" (UID: \"522a4c3d-a66a-4f5f-b8fa-e25cbf48c969\") " pod="openshift-cluster-node-tuning-operator/tuned-vhcmx" Jul 07 11:11:18.126028 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.124356 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d2340b4b-179a-4c0a-9879-13b49897c939-ovn-node-metrics-cert\") pod \"ovnkube-node-tzf62\" (UID: \"d2340b4b-179a-4c0a-9879-13b49897c939\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzf62" Jul 07 11:11:18.126028 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.124533 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/522a4c3d-a66a-4f5f-b8fa-e25cbf48c969-tmp\") pod \"tuned-vhcmx\" (UID: \"522a4c3d-a66a-4f5f-b8fa-e25cbf48c969\") " pod="openshift-cluster-node-tuning-operator/tuned-vhcmx" Jul 07 11:11:18.126028 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.125526 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/663d4847-eafe-43d6-a4b8-2bec15919119-agent-certs\") pod \"konnectivity-agent-4b5qw\" (UID: \"663d4847-eafe-43d6-a4b8-2bec15919119\") " pod="kube-system/konnectivity-agent-4b5qw" Jul 07 11:11:18.130588 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.130566 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-644qj\" (UniqueName: \"kubernetes.io/projected/5e4c9eaf-157e-4ada-8ee8-3d9319eb26a9-kube-api-access-644qj\") pod \"aws-ebs-csi-driver-node-n8tzm\" (UID: \"5e4c9eaf-157e-4ada-8ee8-3d9319eb26a9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n8tzm" Jul 07 11:11:18.133624 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.133586 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ztsrh\" (UniqueName: \"kubernetes.io/projected/6cc36837-424c-44fd-9644-1628e5e5d144-kube-api-access-ztsrh\") pod \"multus-additional-cni-plugins-6h8jc\" (UID: \"6cc36837-424c-44fd-9644-1628e5e5d144\") " pod="openshift-multus/multus-additional-cni-plugins-6h8jc" Jul 07 11:11:18.133751 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.133734 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4jfp9\" (UniqueName: \"kubernetes.io/projected/94c776b2-5ce9-4210-adf5-31efabcef2af-kube-api-access-4jfp9\") pod \"network-metrics-daemon-2q46z\" (UID: \"94c776b2-5ce9-4210-adf5-31efabcef2af\") " pod="openshift-multus/network-metrics-daemon-2q46z" Jul 07 11:11:18.135303 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.135280 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6rsn9\" (UniqueName: \"kubernetes.io/projected/95dba470-f153-4c3c-93d6-c3fb757fec0b-kube-api-access-6rsn9\") pod \"multus-trhvm\" (UID: \"95dba470-f153-4c3c-93d6-c3fb757fec0b\") " pod="openshift-multus/multus-trhvm" Jul 07 11:11:18.135572 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.135556 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7hmhh\" (UniqueName: \"kubernetes.io/projected/522a4c3d-a66a-4f5f-b8fa-e25cbf48c969-kube-api-access-7hmhh\") pod \"tuned-vhcmx\" (UID: \"522a4c3d-a66a-4f5f-b8fa-e25cbf48c969\") " pod="openshift-cluster-node-tuning-operator/tuned-vhcmx" Jul 07 11:11:18.135633 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.135561 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kfm4r\" (UniqueName: \"kubernetes.io/projected/e93d8868-ca25-4c0a-9f25-da7b8c286f60-kube-api-access-kfm4r\") pod \"node-ca-528n2\" (UID: \"e93d8868-ca25-4c0a-9f25-da7b8c286f60\") " pod="openshift-image-registry/node-ca-528n2" Jul 07 11:11:18.137057 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.137030 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tpn9m\" (UniqueName: \"kubernetes.io/projected/d2340b4b-179a-4c0a-9879-13b49897c939-kube-api-access-tpn9m\") pod \"ovnkube-node-tzf62\" (UID: \"d2340b4b-179a-4c0a-9879-13b49897c939\") " pod="openshift-ovn-kubernetes/ovnkube-node-tzf62" Jul 07 11:11:18.193245 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.193228 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-15.ec2.internal" Jul 07 11:11:18.200927 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.200888 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-vrfk8" podStartSLOduration=0.20087591 podStartE2EDuration="200.87591ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 11:11:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:11:18.200732363 +0000 UTC m=+2.614601545" watchObservedRunningTime="2026-07-07 11:11:18.20087591 +0000 UTC m=+2.614745078" Jul 07 11:11:18.214980 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.214954 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-15.ec2.internal" Jul 07 11:11:18.220710 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:18.220686 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-15.ec2.internal\" not found" Jul 07 11:11:18.222055 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.222014 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f5e35157-929c-493a-9da0-386b354a2acb-original-pull-secret\") pod \"global-pull-secret-syncer-vrfk8\" (UID: \"f5e35157-929c-493a-9da0-386b354a2acb\") " pod="kube-system/global-pull-secret-syncer-vrfk8" Jul 07 11:11:18.222055 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.222047 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a9da5840-e6b4-4719-98cd-ef0d46d8470c-tmp-dir\") pod \"node-resolver-77v22\" (UID: \"a9da5840-e6b4-4719-98cd-ef0d46d8470c\") " pod="openshift-dns/node-resolver-77v22" Jul 07 11:11:18.222235 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.222074 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f5e35157-929c-493a-9da0-386b354a2acb-dbus\") pod \"global-pull-secret-syncer-vrfk8\" (UID: \"f5e35157-929c-493a-9da0-386b354a2acb\") " pod="kube-system/global-pull-secret-syncer-vrfk8" Jul 07 11:11:18.222235 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.222117 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/a9da5840-e6b4-4719-98cd-ef0d46d8470c-hosts-file\") pod \"node-resolver-77v22\" (UID: \"a9da5840-e6b4-4719-98cd-ef0d46d8470c\") " pod="openshift-dns/node-resolver-77v22" Jul 07 11:11:18.222235 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.222140 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lw46c\" (UniqueName: \"kubernetes.io/projected/a9da5840-e6b4-4719-98cd-ef0d46d8470c-kube-api-access-lw46c\") pod \"node-resolver-77v22\" (UID: \"a9da5840-e6b4-4719-98cd-ef0d46d8470c\") " pod="openshift-dns/node-resolver-77v22" Jul 07 11:11:18.222235 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.222170 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d4ebce69-4659-439d-b4fa-8e9f4b17ec2a-host-slash\") pod \"iptables-alerter-dvhgx\" (UID: \"d4ebce69-4659-439d-b4fa-8e9f4b17ec2a\") " pod="openshift-network-operator/iptables-alerter-dvhgx" Jul 07 11:11:18.222235 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.222194 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f5e35157-929c-493a-9da0-386b354a2acb-kubelet-config\") pod \"global-pull-secret-syncer-vrfk8\" (UID: \"f5e35157-929c-493a-9da0-386b354a2acb\") " pod="kube-system/global-pull-secret-syncer-vrfk8" Jul 07 11:11:18.222235 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.222220 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gg4ts\" (UniqueName: \"kubernetes.io/projected/d4ebce69-4659-439d-b4fa-8e9f4b17ec2a-kube-api-access-gg4ts\") pod \"iptables-alerter-dvhgx\" (UID: \"d4ebce69-4659-439d-b4fa-8e9f4b17ec2a\") " pod="openshift-network-operator/iptables-alerter-dvhgx" Jul 07 11:11:18.222625 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.222250 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7nvg6\" (UniqueName: \"kubernetes.io/projected/b9ca66a2-4c5a-4816-a081-bb0fecbdab5d-kube-api-access-7nvg6\") pod \"network-check-target-sc5bl\" (UID: \"b9ca66a2-4c5a-4816-a081-bb0fecbdab5d\") " pod="openshift-network-diagnostics/network-check-target-sc5bl" Jul 07 11:11:18.222625 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.222273 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d4ebce69-4659-439d-b4fa-8e9f4b17ec2a-iptables-alerter-script\") pod \"iptables-alerter-dvhgx\" (UID: \"d4ebce69-4659-439d-b4fa-8e9f4b17ec2a\") " pod="openshift-network-operator/iptables-alerter-dvhgx" Jul 07 11:11:18.222625 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.222538 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f5e35157-929c-493a-9da0-386b354a2acb-kubelet-config\") pod \"global-pull-secret-syncer-vrfk8\" (UID: \"f5e35157-929c-493a-9da0-386b354a2acb\") " pod="kube-system/global-pull-secret-syncer-vrfk8" Jul 07 11:11:18.222625 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.222574 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d4ebce69-4659-439d-b4fa-8e9f4b17ec2a-host-slash\") pod \"iptables-alerter-dvhgx\" (UID: \"d4ebce69-4659-439d-b4fa-8e9f4b17ec2a\") " pod="openshift-network-operator/iptables-alerter-dvhgx" Jul 07 11:11:18.222625 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.222560 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/a9da5840-e6b4-4719-98cd-ef0d46d8470c-hosts-file\") pod \"node-resolver-77v22\" (UID: \"a9da5840-e6b4-4719-98cd-ef0d46d8470c\") " pod="openshift-dns/node-resolver-77v22" Jul 07 11:11:18.222625 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.222604 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 11:11:18.222625 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.222618 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 11:11:18.223010 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.222603 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 11:11:18.223010 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.222639 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 11:11:18.223010 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.222702 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f5e35157-929c-493a-9da0-386b354a2acb-dbus\") pod \"global-pull-secret-syncer-vrfk8\" (UID: \"f5e35157-929c-493a-9da0-386b354a2acb\") " pod="kube-system/global-pull-secret-syncer-vrfk8" Jul 07 11:11:18.223010 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.222781 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a9da5840-e6b4-4719-98cd-ef0d46d8470c-tmp-dir\") pod \"node-resolver-77v22\" (UID: \"a9da5840-e6b4-4719-98cd-ef0d46d8470c\") " pod="openshift-dns/node-resolver-77v22" Jul 07 11:11:18.223010 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.222791 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d4ebce69-4659-439d-b4fa-8e9f4b17ec2a-iptables-alerter-script\") pod \"iptables-alerter-dvhgx\" (UID: \"d4ebce69-4659-439d-b4fa-8e9f4b17ec2a\") " pod="openshift-network-operator/iptables-alerter-dvhgx" Jul 07 11:11:18.224288 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.223986 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasSufficientMemory" Jul 07 11:11:18.224288 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.224036 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 11:11:18.224288 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.224054 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasSufficientPID" Jul 07 11:11:18.224487 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.224331 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasSufficientMemory" Jul 07 11:11:18.224487 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.224350 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 11:11:18.224487 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.224364 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasSufficientPID" Jul 07 11:11:18.224631 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.224560 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasSufficientMemory" Jul 07 11:11:18.224631 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.224568 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasSufficientMemory" Jul 07 11:11:18.224631 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.224587 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 11:11:18.224631 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.224602 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasSufficientPID" Jul 07 11:11:18.224631 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.224602 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 11:11:18.224784 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.224627 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasSufficientPID" Jul 07 11:11:18.224784 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:18.224727 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 11:11:18.224784 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:18.224768 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f5e35157-929c-493a-9da0-386b354a2acb-original-pull-secret podName:f5e35157-929c-493a-9da0-386b354a2acb nodeName:}" failed. No retries permitted until 2026-07-07 11:11:18.724753001 +0000 UTC m=+3.138622161 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f5e35157-929c-493a-9da0-386b354a2acb-original-pull-secret") pod "global-pull-secret-syncer-vrfk8" (UID: "f5e35157-929c-493a-9da0-386b354a2acb") : object "kube-system"/"original-pull-secret" not registered Jul 07 11:11:18.229923 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.229904 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-vhcmx" Jul 07 11:11:18.232102 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:18.232085 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 11:11:18.232187 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:18.232106 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 11:11:18.232187 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:18.232118 2562 projected.go:196] Error preparing data for projected volume kube-api-access-7nvg6 for pod openshift-network-diagnostics/network-check-target-sc5bl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 11:11:18.232187 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:18.232176 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b9ca66a2-4c5a-4816-a081-bb0fecbdab5d-kube-api-access-7nvg6 podName:b9ca66a2-4c5a-4816-a081-bb0fecbdab5d nodeName:}" failed. No retries permitted until 2026-07-07 11:11:18.732160535 +0000 UTC m=+3.146029694 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-7nvg6" (UniqueName: "kubernetes.io/projected/b9ca66a2-4c5a-4816-a081-bb0fecbdab5d-kube-api-access-7nvg6") pod "network-check-target-sc5bl" (UID: "b9ca66a2-4c5a-4816-a081-bb0fecbdab5d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 11:11:18.232593 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.232569 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gg4ts\" (UniqueName: \"kubernetes.io/projected/d4ebce69-4659-439d-b4fa-8e9f4b17ec2a-kube-api-access-gg4ts\") pod \"iptables-alerter-dvhgx\" (UID: \"d4ebce69-4659-439d-b4fa-8e9f4b17ec2a\") " pod="openshift-network-operator/iptables-alerter-dvhgx" Jul 07 11:11:18.233945 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.233927 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lw46c\" (UniqueName: \"kubernetes.io/projected/a9da5840-e6b4-4719-98cd-ef0d46d8470c-kube-api-access-lw46c\") pod \"node-resolver-77v22\" (UID: \"a9da5840-e6b4-4719-98cd-ef0d46d8470c\") " pod="openshift-dns/node-resolver-77v22" Jul 07 11:11:18.253785 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.253764 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-6h8jc" Jul 07 11:11:18.259040 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.259021 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-528n2" Jul 07 11:11:18.273219 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.273196 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-4b5qw" Jul 07 11:11:18.279378 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.279355 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-trhvm" Jul 07 11:11:18.301189 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.301161 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-tzf62" Jul 07 11:11:18.317498 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.317474 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n8tzm" Jul 07 11:11:18.320791 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:18.320766 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-15.ec2.internal\" not found" Jul 07 11:11:18.333184 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.333165 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-dvhgx" Jul 07 11:11:18.336863 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.335915 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-77v22" Jul 07 11:11:18.421385 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:18.421366 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-15.ec2.internal\" not found" Jul 07 11:11:18.521471 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:18.521445 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-15.ec2.internal\" not found" Jul 07 11:11:18.621548 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:18.621525 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-15.ec2.internal\" not found" Jul 07 11:11:18.626103 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.626086 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/94c776b2-5ce9-4210-adf5-31efabcef2af-metrics-certs\") pod \"network-metrics-daemon-2q46z\" (UID: \"94c776b2-5ce9-4210-adf5-31efabcef2af\") " pod="openshift-multus/network-metrics-daemon-2q46z" Jul 07 11:11:18.626201 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.626190 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 11:11:18.627347 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.627333 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasSufficientMemory" Jul 07 11:11:18.627425 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.627363 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 11:11:18.627425 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.627379 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasSufficientPID" Jul 07 11:11:18.627496 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:18.627473 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 11:11:18.627533 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:18.627524 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/94c776b2-5ce9-4210-adf5-31efabcef2af-metrics-certs podName:94c776b2-5ce9-4210-adf5-31efabcef2af nodeName:}" failed. No retries permitted until 2026-07-07 11:11:19.627505552 +0000 UTC m=+4.041374708 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/94c776b2-5ce9-4210-adf5-31efabcef2af-metrics-certs") pod "network-metrics-daemon-2q46z" (UID: "94c776b2-5ce9-4210-adf5-31efabcef2af") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 11:11:18.722153 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:18.722096 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-15.ec2.internal\" not found" Jul 07 11:11:18.726442 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.726423 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f5e35157-929c-493a-9da0-386b354a2acb-original-pull-secret\") pod \"global-pull-secret-syncer-vrfk8\" (UID: \"f5e35157-929c-493a-9da0-386b354a2acb\") " pod="kube-system/global-pull-secret-syncer-vrfk8" Jul 07 11:11:18.726594 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.726568 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 11:11:18.728069 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.728051 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasSufficientMemory" Jul 07 11:11:18.728177 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.728089 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 11:11:18.728177 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.728108 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasSufficientPID" Jul 07 11:11:18.728296 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:18.728205 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 11:11:18.728296 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:18.728252 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f5e35157-929c-493a-9da0-386b354a2acb-original-pull-secret podName:f5e35157-929c-493a-9da0-386b354a2acb nodeName:}" failed. No retries permitted until 2026-07-07 11:11:19.728235166 +0000 UTC m=+4.142104311 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f5e35157-929c-493a-9da0-386b354a2acb-original-pull-secret") pod "global-pull-secret-syncer-vrfk8" (UID: "f5e35157-929c-493a-9da0-386b354a2acb") : object "kube-system"/"original-pull-secret" not registered Jul 07 11:11:18.808333 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:18.808286 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddcf962a5f27f6d615415b27111cc76dc.slice/crio-e65aacbb602e263d69b53b77ae49a79b7a1f0a151934cc77b57196631c848de3 WatchSource:0}: Error finding container e65aacbb602e263d69b53b77ae49a79b7a1f0a151934cc77b57196631c848de3: Status 404 returned error can't find the container with id e65aacbb602e263d69b53b77ae49a79b7a1f0a151934cc77b57196631c848de3 Jul 07 11:11:18.808707 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:18.808683 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2340b4b_179a_4c0a_9879_13b49897c939.slice/crio-3ae24543d264fbbc0caa3726f8057cb37dc7137af57280d8ff8e3962cf208424 WatchSource:0}: Error finding container 3ae24543d264fbbc0caa3726f8057cb37dc7137af57280d8ff8e3962cf208424: Status 404 returned error can't find the container with id 3ae24543d264fbbc0caa3726f8057cb37dc7137af57280d8ff8e3962cf208424 Jul 07 11:11:18.809851 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:18.809827 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6cc36837_424c_44fd_9644_1628e5e5d144.slice/crio-bb0370ff45f9de93c5ad1aa2ea8d6998cf8c98766c09060436bfe7c5ab1b327f WatchSource:0}: Error finding container bb0370ff45f9de93c5ad1aa2ea8d6998cf8c98766c09060436bfe7c5ab1b327f: Status 404 returned error can't find the container with id bb0370ff45f9de93c5ad1aa2ea8d6998cf8c98766c09060436bfe7c5ab1b327f Jul 07 11:11:18.818851 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.817105 2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 11:11:18.822894 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:18.822866 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-15.ec2.internal\" not found" Jul 07 11:11:18.826754 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.826733 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7nvg6\" (UniqueName: \"kubernetes.io/projected/b9ca66a2-4c5a-4816-a081-bb0fecbdab5d-kube-api-access-7nvg6\") pod \"network-check-target-sc5bl\" (UID: \"b9ca66a2-4c5a-4816-a081-bb0fecbdab5d\") " pod="openshift-network-diagnostics/network-check-target-sc5bl" Jul 07 11:11:18.826843 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.826808 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 11:11:18.828123 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.828108 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasSufficientMemory" Jul 07 11:11:18.828193 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.828137 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 11:11:18.828193 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:18.828147 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasSufficientPID" Jul 07 11:11:18.828316 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:18.828301 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 11:11:18.828374 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:18.828324 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 11:11:18.828374 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:18.828339 2562 projected.go:196] Error preparing data for projected volume kube-api-access-7nvg6 for pod openshift-network-diagnostics/network-check-target-sc5bl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 11:11:18.828472 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:18.828398 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b9ca66a2-4c5a-4816-a081-bb0fecbdab5d-kube-api-access-7nvg6 podName:b9ca66a2-4c5a-4816-a081-bb0fecbdab5d nodeName:}" failed. No retries permitted until 2026-07-07 11:11:19.828383407 +0000 UTC m=+4.242252551 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-7nvg6" (UniqueName: "kubernetes.io/projected/b9ca66a2-4c5a-4816-a081-bb0fecbdab5d-kube-api-access-7nvg6") pod "network-check-target-sc5bl" (UID: "b9ca66a2-4c5a-4816-a081-bb0fecbdab5d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 11:11:18.831400 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:18.831370 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode93d8868_ca25_4c0a_9f25_da7b8c286f60.slice/crio-b48ce44a0eff10368c68c7e69bac5f102204f5f5060ef2f4ba2dfa8adc2ccef1 WatchSource:0}: Error finding container b48ce44a0eff10368c68c7e69bac5f102204f5f5060ef2f4ba2dfa8adc2ccef1: Status 404 returned error can't find the container with id b48ce44a0eff10368c68c7e69bac5f102204f5f5060ef2f4ba2dfa8adc2ccef1 Jul 07 11:11:18.861255 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:18.861232 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9da5840_e6b4_4719_98cd_ef0d46d8470c.slice/crio-5972bee9675dc6a3804487b631012ba3ada1c47db13a060d36e6f8999ce48e33 WatchSource:0}: Error finding container 5972bee9675dc6a3804487b631012ba3ada1c47db13a060d36e6f8999ce48e33: Status 404 returned error can't find the container with id 5972bee9675dc6a3804487b631012ba3ada1c47db13a060d36e6f8999ce48e33 Jul 07 11:11:18.864571 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:18.864548 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod522a4c3d_a66a_4f5f_b8fa_e25cbf48c969.slice/crio-ee23f354e6767a6a9b025a0b7221c59fc667cce1d355bee51a9e9d0f9318100f WatchSource:0}: Error finding container ee23f354e6767a6a9b025a0b7221c59fc667cce1d355bee51a9e9d0f9318100f: Status 404 returned error can't find the container with id ee23f354e6767a6a9b025a0b7221c59fc667cce1d355bee51a9e9d0f9318100f Jul 07 11:11:18.866585 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:18.866566 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e4c9eaf_157e_4ada_8ee8_3d9319eb26a9.slice/crio-ea317cff0ac620f2b07e652a57346ca88f293ef3993ef2ab997f15ea0811212a WatchSource:0}: Error finding container ea317cff0ac620f2b07e652a57346ca88f293ef3993ef2ab997f15ea0811212a: Status 404 returned error can't find the container with id ea317cff0ac620f2b07e652a57346ca88f293ef3993ef2ab997f15ea0811212a Jul 07 11:11:18.886986 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:18.886965 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0c235cfc97783f979ae1f6fa1e22e58.slice/crio-1ceebb6c398b464b302e5dc0708ce7e95d33b6916aa136fc31784f58180ccf9a WatchSource:0}: Error finding container 1ceebb6c398b464b302e5dc0708ce7e95d33b6916aa136fc31784f58180ccf9a: Status 404 returned error can't find the container with id 1ceebb6c398b464b302e5dc0708ce7e95d33b6916aa136fc31784f58180ccf9a Jul 07 11:11:18.920691 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:18.920672 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95dba470_f153_4c3c_93d6_c3fb757fec0b.slice/crio-89954cbcb50a50dfb6f79030d4c0a048142fa5b6f63e60d105b8a69fd256567a WatchSource:0}: Error finding container 89954cbcb50a50dfb6f79030d4c0a048142fa5b6f63e60d105b8a69fd256567a: Status 404 returned error can't find the container with id 89954cbcb50a50dfb6f79030d4c0a048142fa5b6f63e60d105b8a69fd256567a Jul 07 11:11:18.921078 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:18.921057 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod663d4847_eafe_43d6_a4b8_2bec15919119.slice/crio-5660b2500bd571a4817a94fa05688d905e009860ae471fd00cf6237820842111 WatchSource:0}: Error finding container 5660b2500bd571a4817a94fa05688d905e009860ae471fd00cf6237820842111: Status 404 returned error can't find the container with id 5660b2500bd571a4817a94fa05688d905e009860ae471fd00cf6237820842111 Jul 07 11:11:18.923008 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:18.922988 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-15.ec2.internal\" not found" Jul 07 11:11:18.962078 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:11:18.962056 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4ebce69_4659_439d_b4fa_8e9f4b17ec2a.slice/crio-e99e5c5f35ecab8db00ac08a637c33cbd0842ade6ee9f88428b037106a3cba1d WatchSource:0}: Error finding container e99e5c5f35ecab8db00ac08a637c33cbd0842ade6ee9f88428b037106a3cba1d: Status 404 returned error can't find the container with id e99e5c5f35ecab8db00ac08a637c33cbd0842ade6ee9f88428b037106a3cba1d Jul 07 11:11:18.965176 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:18.965158 2562 kuberuntime_manager.go:1459] "Unhandled Error" err="container iptables-alerter start failed in pod iptables-alerter-dvhgx_openshift-network-operator(d4ebce69-4659-439d-b4fa-8e9f4b17ec2a): ErrImagePull: pull QPS exceeded" logger="UnhandledError" Jul 07 11:11:18.965242 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:18.965188 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"iptables-alerter\" with ErrImagePull: \"pull QPS exceeded\"" pod="openshift-network-operator/iptables-alerter-dvhgx" podUID="d4ebce69-4659-439d-b4fa-8e9f4b17ec2a" Jul 07 11:11:19.023547 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:19.023496 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-15.ec2.internal\" not found" Jul 07 11:11:19.124050 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:19.124025 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-15.ec2.internal\" not found" Jul 07 11:11:19.219513 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:19.219494 2562 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 11:11:19.224997 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:19.224966 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-15.ec2.internal\" not found" Jul 07 11:11:19.325096 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:19.325040 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-15.ec2.internal\" not found" Jul 07 11:11:19.426109 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:19.426080 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-15.ec2.internal\" not found" Jul 07 11:11:19.526220 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:19.526189 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-15.ec2.internal\" not found" Jul 07 11:11:19.627361 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:19.627330 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-15.ec2.internal\" not found" Jul 07 11:11:19.633725 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:19.633698 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/94c776b2-5ce9-4210-adf5-31efabcef2af-metrics-certs\") pod \"network-metrics-daemon-2q46z\" (UID: \"94c776b2-5ce9-4210-adf5-31efabcef2af\") " pod="openshift-multus/network-metrics-daemon-2q46z" Jul 07 11:11:19.633878 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:19.633861 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 11:11:19.636502 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:19.635565 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasSufficientMemory" Jul 07 11:11:19.636502 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:19.635600 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 11:11:19.636502 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:19.635616 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasSufficientPID" Jul 07 11:11:19.636502 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:19.635741 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 11:11:19.636502 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:19.635794 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/94c776b2-5ce9-4210-adf5-31efabcef2af-metrics-certs podName:94c776b2-5ce9-4210-adf5-31efabcef2af nodeName:}" failed. No retries permitted until 2026-07-07 11:11:21.635772657 +0000 UTC m=+6.049641809 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/94c776b2-5ce9-4210-adf5-31efabcef2af-metrics-certs") pod "network-metrics-daemon-2q46z" (UID: "94c776b2-5ce9-4210-adf5-31efabcef2af") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 11:11:19.727915 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:19.727774 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-15.ec2.internal\" not found" Jul 07 11:11:19.734354 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:19.734286 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f5e35157-929c-493a-9da0-386b354a2acb-original-pull-secret\") pod \"global-pull-secret-syncer-vrfk8\" (UID: \"f5e35157-929c-493a-9da0-386b354a2acb\") " pod="kube-system/global-pull-secret-syncer-vrfk8" Jul 07 11:11:19.734482 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:19.734409 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 11:11:19.736931 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:19.736890 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasSufficientMemory" Jul 07 11:11:19.736931 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:19.736940 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 11:11:19.737130 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:19.736957 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasSufficientPID" Jul 07 11:11:19.737130 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:19.737113 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 11:11:19.737234 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:19.737174 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f5e35157-929c-493a-9da0-386b354a2acb-original-pull-secret podName:f5e35157-929c-493a-9da0-386b354a2acb nodeName:}" failed. No retries permitted until 2026-07-07 11:11:21.737152329 +0000 UTC m=+6.151021499 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f5e35157-929c-493a-9da0-386b354a2acb-original-pull-secret") pod "global-pull-secret-syncer-vrfk8" (UID: "f5e35157-929c-493a-9da0-386b354a2acb") : object "kube-system"/"original-pull-secret" not registered Jul 07 11:11:19.763774 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:19.763747 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2q46z" Jul 07 11:11:19.763885 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:19.763797 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 11:11:19.764039 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:19.763983 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sc5bl" Jul 07 11:11:19.764039 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:19.764029 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 11:11:19.766120 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:19.766099 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasSufficientMemory" Jul 07 11:11:19.766225 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:19.766139 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 11:11:19.766225 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:19.766156 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasSufficientPID" Jul 07 11:11:19.766401 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:19.766377 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sc5bl" podUID="b9ca66a2-4c5a-4816-a081-bb0fecbdab5d" Jul 07 11:11:19.767191 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:19.767168 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasSufficientMemory" Jul 07 11:11:19.767284 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:19.767206 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 11:11:19.767284 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:19.767222 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeHasSufficientPID" Jul 07 11:11:19.767414 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:19.767392 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2q46z" podUID="94c776b2-5ce9-4210-adf5-31efabcef2af" Jul 07 11:11:19.778284 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:19.778214 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-77v22" event={"ID":"a9da5840-e6b4-4719-98cd-ef0d46d8470c","Type":"ContainerStarted","Data":"5972bee9675dc6a3804487b631012ba3ada1c47db13a060d36e6f8999ce48e33"} Jul 07 11:11:19.789236 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:19.789208 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tzf62" event={"ID":"d2340b4b-179a-4c0a-9879-13b49897c939","Type":"ContainerStarted","Data":"3ae24543d264fbbc0caa3726f8057cb37dc7137af57280d8ff8e3962cf208424"} Jul 07 11:11:19.790840 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:19.790817 2562 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 11:11:19.805300 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:19.805275 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-trhvm" event={"ID":"95dba470-f153-4c3c-93d6-c3fb757fec0b","Type":"ContainerStarted","Data":"89954cbcb50a50dfb6f79030d4c0a048142fa5b6f63e60d105b8a69fd256567a"} Jul 07 11:11:19.813499 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:19.813418 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-vhcmx" event={"ID":"522a4c3d-a66a-4f5f-b8fa-e25cbf48c969","Type":"ContainerStarted","Data":"ee23f354e6767a6a9b025a0b7221c59fc667cce1d355bee51a9e9d0f9318100f"} Jul 07 11:11:19.814973 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:19.814785 2562 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-15.ec2.internal" Jul 07 11:11:19.819978 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:19.819895 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-528n2" event={"ID":"e93d8868-ca25-4c0a-9f25-da7b8c286f60","Type":"ContainerStarted","Data":"b48ce44a0eff10368c68c7e69bac5f102204f5f5060ef2f4ba2dfa8adc2ccef1"} Jul 07 11:11:19.828035 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:19.827662 2562 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 07 11:11:19.828035 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:19.827754 2562 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-15.ec2.internal" Jul 07 11:11:19.828778 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:19.828745 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-15.ec2.internal"] Jul 07 11:11:19.832109 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:19.832085 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6h8jc" event={"ID":"6cc36837-424c-44fd-9644-1628e5e5d144","Type":"ContainerStarted","Data":"bb0370ff45f9de93c5ad1aa2ea8d6998cf8c98766c09060436bfe7c5ab1b327f"} Jul 07 11:11:19.834860 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:19.834838 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f5e35157-929c-493a-9da0-386b354a2acb-kubelet-config\") pod \"f5e35157-929c-493a-9da0-386b354a2acb\" (UID: \"f5e35157-929c-493a-9da0-386b354a2acb\") " Jul 07 11:11:19.834967 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:19.834908 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f5e35157-929c-493a-9da0-386b354a2acb-dbus\") pod \"f5e35157-929c-493a-9da0-386b354a2acb\" (UID: \"f5e35157-929c-493a-9da0-386b354a2acb\") " Jul 07 11:11:19.835052 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:19.834989 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7nvg6\" (UniqueName: \"kubernetes.io/projected/b9ca66a2-4c5a-4816-a081-bb0fecbdab5d-kube-api-access-7nvg6\") pod \"network-check-target-sc5bl\" (UID: \"b9ca66a2-4c5a-4816-a081-bb0fecbdab5d\") " pod="openshift-network-diagnostics/network-check-target-sc5bl" Jul 07 11:11:19.835208 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:19.835191 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 11:11:19.835273 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:19.835218 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 11:11:19.835273 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:19.835232 2562 projected.go:196] Error preparing data for projected volume kube-api-access-7nvg6 for pod openshift-network-diagnostics/network-check-target-sc5bl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 11:11:19.835375 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:19.835283 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b9ca66a2-4c5a-4816-a081-bb0fecbdab5d-kube-api-access-7nvg6 podName:b9ca66a2-4c5a-4816-a081-bb0fecbdab5d nodeName:}" failed. No retries permitted until 2026-07-07 11:11:21.835265503 +0000 UTC m=+6.249134671 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-7nvg6" (UniqueName: "kubernetes.io/projected/b9ca66a2-4c5a-4816-a081-bb0fecbdab5d-kube-api-access-7nvg6") pod "network-check-target-sc5bl" (UID: "b9ca66a2-4c5a-4816-a081-bb0fecbdab5d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 11:11:19.835599 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:19.835575 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/f5e35157-929c-493a-9da0-386b354a2acb-kubelet-config" (OuterVolumeSpecName: "kubelet-config") pod "f5e35157-929c-493a-9da0-386b354a2acb" (UID: "f5e35157-929c-493a-9da0-386b354a2acb"). InnerVolumeSpecName "kubelet-config". PluginName "kubernetes.io/host-path", VolumeGIDValue "" Jul 07 11:11:19.835673 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:19.835623 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/host-path/f5e35157-929c-493a-9da0-386b354a2acb-dbus" (OuterVolumeSpecName: "dbus") pod "f5e35157-929c-493a-9da0-386b354a2acb" (UID: "f5e35157-929c-493a-9da0-386b354a2acb"). InnerVolumeSpecName "dbus". PluginName "kubernetes.io/host-path", VolumeGIDValue "" Jul 07 11:11:19.838572 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:19.838549 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-136-15.ec2.internal"] Jul 07 11:11:19.838715 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:19.838698 2562 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 07 11:11:19.839950 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:19.839922 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-15.ec2.internal" event={"ID":"dcf962a5f27f6d615415b27111cc76dc","Type":"ContainerStarted","Data":"e65aacbb602e263d69b53b77ae49a79b7a1f0a151934cc77b57196631c848de3"} Jul 07 11:11:19.852347 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:19.851809 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-dvhgx" event={"ID":"d4ebce69-4659-439d-b4fa-8e9f4b17ec2a","Type":"ContainerStarted","Data":"e99e5c5f35ecab8db00ac08a637c33cbd0842ade6ee9f88428b037106a3cba1d"} Jul 07 11:11:19.866005 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:19.865971 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"iptables-alerter\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:28389dd118629aa9d7c2525202f3ad6359afb98b6a3bf7672cb1ecfae2418e23\\\": ErrImagePull: pull QPS exceeded\"" pod="openshift-network-operator/iptables-alerter-dvhgx" podUID="d4ebce69-4659-439d-b4fa-8e9f4b17ec2a" Jul 07 11:11:19.879236 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:19.878852 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-4b5qw" event={"ID":"663d4847-eafe-43d6-a4b8-2bec15919119","Type":"ContainerStarted","Data":"5660b2500bd571a4817a94fa05688d905e009860ae471fd00cf6237820842111"} Jul 07 11:11:19.893352 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:19.893300 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-15.ec2.internal" event={"ID":"a0c235cfc97783f979ae1f6fa1e22e58","Type":"ContainerStarted","Data":"1ceebb6c398b464b302e5dc0708ce7e95d33b6916aa136fc31784f58180ccf9a"} Jul 07 11:11:19.909662 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:19.909635 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n8tzm" event={"ID":"5e4c9eaf-157e-4ada-8ee8-3d9319eb26a9","Type":"ContainerStarted","Data":"ea317cff0ac620f2b07e652a57346ca88f293ef3993ef2ab997f15ea0811212a"} Jul 07 11:11:19.918003 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:19.917966 2562 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 07 11:11:19.933270 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:19.933247 2562 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 07 11:11:19.935701 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:19.935561 2562 reconciler_common.go:299] "Volume detached for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f5e35157-929c-493a-9da0-386b354a2acb-dbus\") on node \"ip-10-0-136-15.ec2.internal\" DevicePath \"\"" Jul 07 11:11:19.935701 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:19.935583 2562 reconciler_common.go:299] "Volume detached for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f5e35157-929c-493a-9da0-386b354a2acb-kubelet-config\") on node \"ip-10-0-136-15.ec2.internal\" DevicePath \"\"" Jul 07 11:11:19.960344 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:19.960291 2562 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-dhsfx" Jul 07 11:11:19.968314 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:19.968293 2562 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-dhsfx" Jul 07 11:11:20.972200 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:20.972151 2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-06 11:06:20 +0000 UTC" deadline="2027-12-17 14:11:39.866286551 +0000 UTC" Jul 07 11:11:20.972200 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:20.972199 2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12675h0m18.894092332s" Jul 07 11:11:21.650119 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:21.650081 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/94c776b2-5ce9-4210-adf5-31efabcef2af-metrics-certs\") pod \"network-metrics-daemon-2q46z\" (UID: \"94c776b2-5ce9-4210-adf5-31efabcef2af\") " pod="openshift-multus/network-metrics-daemon-2q46z" Jul 07 11:11:21.650331 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:21.650268 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 11:11:21.650412 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:21.650339 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/94c776b2-5ce9-4210-adf5-31efabcef2af-metrics-certs podName:94c776b2-5ce9-4210-adf5-31efabcef2af nodeName:}" failed. No retries permitted until 2026-07-07 11:11:25.650318846 +0000 UTC m=+10.064188007 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/94c776b2-5ce9-4210-adf5-31efabcef2af-metrics-certs") pod "network-metrics-daemon-2q46z" (UID: "94c776b2-5ce9-4210-adf5-31efabcef2af") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 11:11:21.750892 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:21.750857 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f5e35157-929c-493a-9da0-386b354a2acb-original-pull-secret\") pod \"global-pull-secret-syncer-vrfk8\" (UID: \"f5e35157-929c-493a-9da0-386b354a2acb\") " pod="kube-system/global-pull-secret-syncer-vrfk8" Jul 07 11:11:21.751102 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:21.751071 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 11:11:21.751209 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:21.751156 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f5e35157-929c-493a-9da0-386b354a2acb-original-pull-secret podName:f5e35157-929c-493a-9da0-386b354a2acb nodeName:}" failed. No retries permitted until 2026-07-07 11:11:25.751127208 +0000 UTC m=+10.164996366 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f5e35157-929c-493a-9da0-386b354a2acb-original-pull-secret") pod "global-pull-secret-syncer-vrfk8" (UID: "f5e35157-929c-493a-9da0-386b354a2acb") : object "kube-system"/"original-pull-secret" not registered Jul 07 11:11:21.762773 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:21.762747 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2q46z" Jul 07 11:11:21.762972 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:21.762912 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2q46z" podUID="94c776b2-5ce9-4210-adf5-31efabcef2af" Jul 07 11:11:21.763122 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:21.763104 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sc5bl" Jul 07 11:11:21.763231 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:21.763206 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sc5bl" podUID="b9ca66a2-4c5a-4816-a081-bb0fecbdab5d" Jul 07 11:11:21.852293 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:21.852181 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7nvg6\" (UniqueName: \"kubernetes.io/projected/b9ca66a2-4c5a-4816-a081-bb0fecbdab5d-kube-api-access-7nvg6\") pod \"network-check-target-sc5bl\" (UID: \"b9ca66a2-4c5a-4816-a081-bb0fecbdab5d\") " pod="openshift-network-diagnostics/network-check-target-sc5bl" Jul 07 11:11:21.852455 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:21.852390 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 11:11:21.852455 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:21.852428 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 11:11:21.852455 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:21.852443 2562 projected.go:196] Error preparing data for projected volume kube-api-access-7nvg6 for pod openshift-network-diagnostics/network-check-target-sc5bl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 11:11:21.852625 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:21.852509 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b9ca66a2-4c5a-4816-a081-bb0fecbdab5d-kube-api-access-7nvg6 podName:b9ca66a2-4c5a-4816-a081-bb0fecbdab5d nodeName:}" failed. No retries permitted until 2026-07-07 11:11:25.852489296 +0000 UTC m=+10.266358448 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-7nvg6" (UniqueName: "kubernetes.io/projected/b9ca66a2-4c5a-4816-a081-bb0fecbdab5d-kube-api-access-7nvg6") pod "network-check-target-sc5bl" (UID: "b9ca66a2-4c5a-4816-a081-bb0fecbdab5d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 11:11:21.972681 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:21.972582 2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-06 11:06:20 +0000 UTC" deadline="2027-12-25 13:31:44.022640376 +0000 UTC" Jul 07 11:11:21.972681 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:21.972620 2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12866h20m22.050024542s" Jul 07 11:11:23.761480 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:23.760934 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sc5bl" Jul 07 11:11:23.761480 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:23.761075 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sc5bl" podUID="b9ca66a2-4c5a-4816-a081-bb0fecbdab5d" Jul 07 11:11:23.761480 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:23.761125 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2q46z" Jul 07 11:11:23.761480 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:23.761253 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2q46z" podUID="94c776b2-5ce9-4210-adf5-31efabcef2af" Jul 07 11:11:25.681437 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:25.681406 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/94c776b2-5ce9-4210-adf5-31efabcef2af-metrics-certs\") pod \"network-metrics-daemon-2q46z\" (UID: \"94c776b2-5ce9-4210-adf5-31efabcef2af\") " pod="openshift-multus/network-metrics-daemon-2q46z" Jul 07 11:11:25.681881 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:25.681579 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 11:11:25.681881 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:25.681659 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/94c776b2-5ce9-4210-adf5-31efabcef2af-metrics-certs podName:94c776b2-5ce9-4210-adf5-31efabcef2af nodeName:}" failed. No retries permitted until 2026-07-07 11:11:33.681638002 +0000 UTC m=+18.095507163 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/94c776b2-5ce9-4210-adf5-31efabcef2af-metrics-certs") pod "network-metrics-daemon-2q46z" (UID: "94c776b2-5ce9-4210-adf5-31efabcef2af") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 11:11:25.761618 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:25.760973 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2q46z" Jul 07 11:11:25.761618 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:25.760984 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sc5bl" Jul 07 11:11:25.761618 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:25.761105 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2q46z" podUID="94c776b2-5ce9-4210-adf5-31efabcef2af" Jul 07 11:11:25.761618 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:25.761534 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sc5bl" podUID="b9ca66a2-4c5a-4816-a081-bb0fecbdab5d" Jul 07 11:11:25.782000 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:25.781972 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f5e35157-929c-493a-9da0-386b354a2acb-original-pull-secret\") pod \"global-pull-secret-syncer-vrfk8\" (UID: \"f5e35157-929c-493a-9da0-386b354a2acb\") " pod="kube-system/global-pull-secret-syncer-vrfk8" Jul 07 11:11:25.782142 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:25.782094 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 11:11:25.782202 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:25.782151 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f5e35157-929c-493a-9da0-386b354a2acb-original-pull-secret podName:f5e35157-929c-493a-9da0-386b354a2acb nodeName:}" failed. No retries permitted until 2026-07-07 11:11:33.782133193 +0000 UTC m=+18.196002357 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f5e35157-929c-493a-9da0-386b354a2acb-original-pull-secret") pod "global-pull-secret-syncer-vrfk8" (UID: "f5e35157-929c-493a-9da0-386b354a2acb") : object "kube-system"/"original-pull-secret" not registered Jul 07 11:11:25.883190 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:25.883093 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7nvg6\" (UniqueName: \"kubernetes.io/projected/b9ca66a2-4c5a-4816-a081-bb0fecbdab5d-kube-api-access-7nvg6\") pod \"network-check-target-sc5bl\" (UID: \"b9ca66a2-4c5a-4816-a081-bb0fecbdab5d\") " pod="openshift-network-diagnostics/network-check-target-sc5bl" Jul 07 11:11:25.883349 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:25.883234 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 11:11:25.883349 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:25.883260 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 11:11:25.883349 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:25.883277 2562 projected.go:196] Error preparing data for projected volume kube-api-access-7nvg6 for pod openshift-network-diagnostics/network-check-target-sc5bl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 11:11:25.883349 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:25.883333 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b9ca66a2-4c5a-4816-a081-bb0fecbdab5d-kube-api-access-7nvg6 podName:b9ca66a2-4c5a-4816-a081-bb0fecbdab5d nodeName:}" failed. No retries permitted until 2026-07-07 11:11:33.883316019 +0000 UTC m=+18.297185175 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-7nvg6" (UniqueName: "kubernetes.io/projected/b9ca66a2-4c5a-4816-a081-bb0fecbdab5d-kube-api-access-7nvg6") pod "network-check-target-sc5bl" (UID: "b9ca66a2-4c5a-4816-a081-bb0fecbdab5d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 11:11:27.765101 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:27.765073 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sc5bl" Jul 07 11:11:27.765628 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:27.765600 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2q46z" Jul 07 11:11:27.765743 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:27.765718 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sc5bl" podUID="b9ca66a2-4c5a-4816-a081-bb0fecbdab5d" Jul 07 11:11:27.765894 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:27.765861 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2q46z" podUID="94c776b2-5ce9-4210-adf5-31efabcef2af" Jul 07 11:11:27.792619 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:27.792137 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kube-system/global-pull-secret-syncer-vrfk8"] Jul 07 11:11:27.792619 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:27.792232 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vrfk8" Jul 07 11:11:27.797371 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:27.797345 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kube-system/global-pull-secret-syncer-vrfk8"] Jul 07 11:11:27.800202 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:27.800151 2562 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"f5e35157-929c-493a-9da0-386b354a2acb\\\"},\\\"status\\\":{\\\"conditions\\\":[{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2026-07-07T11:11:27Z\\\",\\\"observedGeneration\\\":2,\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2026-07-07T11:11:27Z\\\",\\\"observedGeneration\\\":2,\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"Initialized\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2026-07-07T11:11:27Z\\\",\\\"observedGeneration\\\":2,\\\"reason\\\":\\\"PodFailed\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2026-07-07T11:11:27Z\\\",\\\"observedGeneration\\\":2,\\\"reason\\\":\\\"PodFailed\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"ContainersReady\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2026-07-07T11:11:27Z\\\",\\\"observedGeneration\\\":2,\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"PodScheduled\\\"}],\\\"containerStatuses\\\":[{\\\"image\\\":\\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:2d5a37efb4a917780398d375b7156193497f4554ae216ef110a445724d665111\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"global-pull-secret-syncer\\\",\\\"ready\\\":false,\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"ContainerCreating\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/var/lib/kubelet\\\",\\\"name\\\":\\\"kubelet-config\\\"},{\\\"mountPath\\\":\\\"/var/run/dbus\\\",\\\"name\\\":\\\"dbus\\\"},{\\\"mountPath\\\":\\\"/etc/original-pull-secret\\\",\\\"name\\\":\\\"original-pull-secret\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}],\\\"hostIP\\\":\\\"10.0.136.15\\\",\\\"hostIPs\\\":[{\\\"ip\\\":\\\"10.0.136.15\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kube-system\"/\"global-pull-secret-syncer-vrfk8\": pods \"global-pull-secret-syncer-vrfk8\" not found" pod="kube-system/global-pull-secret-syncer-vrfk8" Jul 07 11:11:27.802148 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:27.802104 2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"global-pull-secret-syncer-vrfk8\" is forbidden: User \"system:node:ip-10-0-136-15.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kube-system\": no relationship found between node 'ip-10-0-136-15.ec2.internal' and this object" podUID="f5e35157-929c-493a-9da0-386b354a2acb" pod="kube-system/global-pull-secret-syncer-vrfk8" Jul 07 11:11:27.826716 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:27.826694 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-7ml6d"] Jul 07 11:11:27.830814 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:27.830792 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7ml6d" Jul 07 11:11:27.830934 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:27.830871 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7ml6d" podUID="a4b1e5f7-71c6-4349-8dad-ef5763f9bfb9" Jul 07 11:11:27.832896 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:27.832869 2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"global-pull-secret-syncer-vrfk8\" is forbidden: User \"system:node:ip-10-0-136-15.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kube-system\": no relationship found between node 'ip-10-0-136-15.ec2.internal' and this object" podUID="f5e35157-929c-493a-9da0-386b354a2acb" pod="kube-system/global-pull-secret-syncer-vrfk8" Jul 07 11:11:27.899706 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:27.899668 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a4b1e5f7-71c6-4349-8dad-ef5763f9bfb9-original-pull-secret\") pod \"global-pull-secret-syncer-7ml6d\" (UID: \"a4b1e5f7-71c6-4349-8dad-ef5763f9bfb9\") " pod="kube-system/global-pull-secret-syncer-7ml6d" Jul 07 11:11:27.899871 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:27.899718 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a4b1e5f7-71c6-4349-8dad-ef5763f9bfb9-kubelet-config\") pod \"global-pull-secret-syncer-7ml6d\" (UID: \"a4b1e5f7-71c6-4349-8dad-ef5763f9bfb9\") " pod="kube-system/global-pull-secret-syncer-7ml6d" Jul 07 11:11:27.899871 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:27.899758 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a4b1e5f7-71c6-4349-8dad-ef5763f9bfb9-dbus\") pod \"global-pull-secret-syncer-7ml6d\" (UID: \"a4b1e5f7-71c6-4349-8dad-ef5763f9bfb9\") " pod="kube-system/global-pull-secret-syncer-7ml6d" Jul 07 11:11:27.899871 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:27.899793 2562 reconciler_common.go:299] "Volume detached for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f5e35157-929c-493a-9da0-386b354a2acb-original-pull-secret\") on node \"ip-10-0-136-15.ec2.internal\" DevicePath \"\"" Jul 07 11:11:28.000528 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:28.000484 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a4b1e5f7-71c6-4349-8dad-ef5763f9bfb9-original-pull-secret\") pod \"global-pull-secret-syncer-7ml6d\" (UID: \"a4b1e5f7-71c6-4349-8dad-ef5763f9bfb9\") " pod="kube-system/global-pull-secret-syncer-7ml6d" Jul 07 11:11:28.000528 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:28.000541 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a4b1e5f7-71c6-4349-8dad-ef5763f9bfb9-kubelet-config\") pod \"global-pull-secret-syncer-7ml6d\" (UID: \"a4b1e5f7-71c6-4349-8dad-ef5763f9bfb9\") " pod="kube-system/global-pull-secret-syncer-7ml6d" Jul 07 11:11:28.000766 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:28.000575 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a4b1e5f7-71c6-4349-8dad-ef5763f9bfb9-dbus\") pod \"global-pull-secret-syncer-7ml6d\" (UID: \"a4b1e5f7-71c6-4349-8dad-ef5763f9bfb9\") " pod="kube-system/global-pull-secret-syncer-7ml6d" Jul 07 11:11:28.000766 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:28.000741 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a4b1e5f7-71c6-4349-8dad-ef5763f9bfb9-dbus\") pod \"global-pull-secret-syncer-7ml6d\" (UID: \"a4b1e5f7-71c6-4349-8dad-ef5763f9bfb9\") " pod="kube-system/global-pull-secret-syncer-7ml6d" Jul 07 11:11:28.000870 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:28.000848 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 11:11:28.001059 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:28.000934 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a4b1e5f7-71c6-4349-8dad-ef5763f9bfb9-original-pull-secret podName:a4b1e5f7-71c6-4349-8dad-ef5763f9bfb9 nodeName:}" failed. No retries permitted until 2026-07-07 11:11:28.500899666 +0000 UTC m=+12.914768814 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a4b1e5f7-71c6-4349-8dad-ef5763f9bfb9-original-pull-secret") pod "global-pull-secret-syncer-7ml6d" (UID: "a4b1e5f7-71c6-4349-8dad-ef5763f9bfb9") : object "kube-system"/"original-pull-secret" not registered Jul 07 11:11:28.001059 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:28.001017 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a4b1e5f7-71c6-4349-8dad-ef5763f9bfb9-kubelet-config\") pod \"global-pull-secret-syncer-7ml6d\" (UID: \"a4b1e5f7-71c6-4349-8dad-ef5763f9bfb9\") " pod="kube-system/global-pull-secret-syncer-7ml6d" Jul 07 11:11:28.504338 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:28.504185 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a4b1e5f7-71c6-4349-8dad-ef5763f9bfb9-original-pull-secret\") pod \"global-pull-secret-syncer-7ml6d\" (UID: \"a4b1e5f7-71c6-4349-8dad-ef5763f9bfb9\") " pod="kube-system/global-pull-secret-syncer-7ml6d" Jul 07 11:11:28.504338 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:28.504328 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 11:11:28.504569 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:28.504391 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a4b1e5f7-71c6-4349-8dad-ef5763f9bfb9-original-pull-secret podName:a4b1e5f7-71c6-4349-8dad-ef5763f9bfb9 nodeName:}" failed. No retries permitted until 2026-07-07 11:11:29.504370912 +0000 UTC m=+13.918240067 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a4b1e5f7-71c6-4349-8dad-ef5763f9bfb9-original-pull-secret") pod "global-pull-secret-syncer-7ml6d" (UID: "a4b1e5f7-71c6-4349-8dad-ef5763f9bfb9") : object "kube-system"/"original-pull-secret" not registered Jul 07 11:11:29.513140 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:29.513106 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a4b1e5f7-71c6-4349-8dad-ef5763f9bfb9-original-pull-secret\") pod \"global-pull-secret-syncer-7ml6d\" (UID: \"a4b1e5f7-71c6-4349-8dad-ef5763f9bfb9\") " pod="kube-system/global-pull-secret-syncer-7ml6d" Jul 07 11:11:29.513570 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:29.513222 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 11:11:29.513570 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:29.513278 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a4b1e5f7-71c6-4349-8dad-ef5763f9bfb9-original-pull-secret podName:a4b1e5f7-71c6-4349-8dad-ef5763f9bfb9 nodeName:}" failed. No retries permitted until 2026-07-07 11:11:31.513265317 +0000 UTC m=+15.927134459 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a4b1e5f7-71c6-4349-8dad-ef5763f9bfb9-original-pull-secret") pod "global-pull-secret-syncer-7ml6d" (UID: "a4b1e5f7-71c6-4349-8dad-ef5763f9bfb9") : object "kube-system"/"original-pull-secret" not registered Jul 07 11:11:29.761110 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:29.761077 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7ml6d" Jul 07 11:11:29.761110 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:29.761096 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2q46z" Jul 07 11:11:29.761340 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:29.761218 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sc5bl" Jul 07 11:11:29.761340 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:29.761227 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2q46z" podUID="94c776b2-5ce9-4210-adf5-31efabcef2af" Jul 07 11:11:29.761340 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:29.761325 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7ml6d" podUID="a4b1e5f7-71c6-4349-8dad-ef5763f9bfb9" Jul 07 11:11:29.761488 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:29.761437 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sc5bl" podUID="b9ca66a2-4c5a-4816-a081-bb0fecbdab5d" Jul 07 11:11:31.527247 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:31.527209 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a4b1e5f7-71c6-4349-8dad-ef5763f9bfb9-original-pull-secret\") pod \"global-pull-secret-syncer-7ml6d\" (UID: \"a4b1e5f7-71c6-4349-8dad-ef5763f9bfb9\") " pod="kube-system/global-pull-secret-syncer-7ml6d" Jul 07 11:11:31.527655 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:31.527349 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 11:11:31.527655 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:31.527410 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a4b1e5f7-71c6-4349-8dad-ef5763f9bfb9-original-pull-secret podName:a4b1e5f7-71c6-4349-8dad-ef5763f9bfb9 nodeName:}" failed. No retries permitted until 2026-07-07 11:11:35.527393152 +0000 UTC m=+19.941262306 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a4b1e5f7-71c6-4349-8dad-ef5763f9bfb9-original-pull-secret") pod "global-pull-secret-syncer-7ml6d" (UID: "a4b1e5f7-71c6-4349-8dad-ef5763f9bfb9") : object "kube-system"/"original-pull-secret" not registered Jul 07 11:11:31.760869 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:31.760839 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7ml6d" Jul 07 11:11:31.761050 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:31.760849 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2q46z" Jul 07 11:11:31.761050 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:31.760950 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7ml6d" podUID="a4b1e5f7-71c6-4349-8dad-ef5763f9bfb9" Jul 07 11:11:31.761050 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:31.760960 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sc5bl" Jul 07 11:11:31.761202 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:31.761074 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2q46z" podUID="94c776b2-5ce9-4210-adf5-31efabcef2af" Jul 07 11:11:31.761202 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:31.761163 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sc5bl" podUID="b9ca66a2-4c5a-4816-a081-bb0fecbdab5d" Jul 07 11:11:33.744844 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:33.744813 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/94c776b2-5ce9-4210-adf5-31efabcef2af-metrics-certs\") pod \"network-metrics-daemon-2q46z\" (UID: \"94c776b2-5ce9-4210-adf5-31efabcef2af\") " pod="openshift-multus/network-metrics-daemon-2q46z" Jul 07 11:11:33.745237 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:33.744981 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 11:11:33.745237 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:33.745049 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/94c776b2-5ce9-4210-adf5-31efabcef2af-metrics-certs podName:94c776b2-5ce9-4210-adf5-31efabcef2af nodeName:}" failed. No retries permitted until 2026-07-07 11:11:49.745033285 +0000 UTC m=+34.158902433 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/94c776b2-5ce9-4210-adf5-31efabcef2af-metrics-certs") pod "network-metrics-daemon-2q46z" (UID: "94c776b2-5ce9-4210-adf5-31efabcef2af") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 11:11:33.763987 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:33.763962 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7ml6d" Jul 07 11:11:33.764107 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:33.763999 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sc5bl" Jul 07 11:11:33.764107 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:33.763999 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2q46z" Jul 07 11:11:33.764107 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:33.764083 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7ml6d" podUID="a4b1e5f7-71c6-4349-8dad-ef5763f9bfb9" Jul 07 11:11:33.764233 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:33.764157 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sc5bl" podUID="b9ca66a2-4c5a-4816-a081-bb0fecbdab5d" Jul 07 11:11:33.764275 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:33.764241 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2q46z" podUID="94c776b2-5ce9-4210-adf5-31efabcef2af" Jul 07 11:11:33.946256 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:33.946220 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7nvg6\" (UniqueName: \"kubernetes.io/projected/b9ca66a2-4c5a-4816-a081-bb0fecbdab5d-kube-api-access-7nvg6\") pod \"network-check-target-sc5bl\" (UID: \"b9ca66a2-4c5a-4816-a081-bb0fecbdab5d\") " pod="openshift-network-diagnostics/network-check-target-sc5bl" Jul 07 11:11:33.946420 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:33.946375 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 11:11:33.946420 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:33.946398 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 11:11:33.946420 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:33.946409 2562 projected.go:196] Error preparing data for projected volume kube-api-access-7nvg6 for pod openshift-network-diagnostics/network-check-target-sc5bl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 11:11:33.946575 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:33.946473 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b9ca66a2-4c5a-4816-a081-bb0fecbdab5d-kube-api-access-7nvg6 podName:b9ca66a2-4c5a-4816-a081-bb0fecbdab5d nodeName:}" failed. No retries permitted until 2026-07-07 11:11:49.946451656 +0000 UTC m=+34.360320821 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-7nvg6" (UniqueName: "kubernetes.io/projected/b9ca66a2-4c5a-4816-a081-bb0fecbdab5d-kube-api-access-7nvg6") pod "network-check-target-sc5bl" (UID: "b9ca66a2-4c5a-4816-a081-bb0fecbdab5d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 11:11:35.558432 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:35.558396 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a4b1e5f7-71c6-4349-8dad-ef5763f9bfb9-original-pull-secret\") pod \"global-pull-secret-syncer-7ml6d\" (UID: \"a4b1e5f7-71c6-4349-8dad-ef5763f9bfb9\") " pod="kube-system/global-pull-secret-syncer-7ml6d" Jul 07 11:11:35.558884 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:35.558564 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 11:11:35.558884 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:35.558645 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a4b1e5f7-71c6-4349-8dad-ef5763f9bfb9-original-pull-secret podName:a4b1e5f7-71c6-4349-8dad-ef5763f9bfb9 nodeName:}" failed. No retries permitted until 2026-07-07 11:11:43.558624367 +0000 UTC m=+27.972493525 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a4b1e5f7-71c6-4349-8dad-ef5763f9bfb9-original-pull-secret") pod "global-pull-secret-syncer-7ml6d" (UID: "a4b1e5f7-71c6-4349-8dad-ef5763f9bfb9") : object "kube-system"/"original-pull-secret" not registered Jul 07 11:11:35.761097 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:35.760904 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sc5bl" Jul 07 11:11:35.761264 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:35.760904 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2q46z" Jul 07 11:11:35.761264 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:35.761188 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sc5bl" podUID="b9ca66a2-4c5a-4816-a081-bb0fecbdab5d" Jul 07 11:11:35.761354 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:35.761272 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2q46z" podUID="94c776b2-5ce9-4210-adf5-31efabcef2af" Jul 07 11:11:35.761354 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:35.760966 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7ml6d" Jul 07 11:11:35.761432 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:35.761359 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7ml6d" podUID="a4b1e5f7-71c6-4349-8dad-ef5763f9bfb9" Jul 07 11:11:37.761283 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:37.761254 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7ml6d" Jul 07 11:11:37.761715 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:37.761351 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7ml6d" podUID="a4b1e5f7-71c6-4349-8dad-ef5763f9bfb9" Jul 07 11:11:37.761715 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:37.761446 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2q46z" Jul 07 11:11:37.761715 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:37.761567 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2q46z" podUID="94c776b2-5ce9-4210-adf5-31efabcef2af" Jul 07 11:11:37.761715 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:37.761617 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sc5bl" Jul 07 11:11:37.761715 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:37.761692 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sc5bl" podUID="b9ca66a2-4c5a-4816-a081-bb0fecbdab5d" Jul 07 11:11:39.761458 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:39.761289 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2q46z" Jul 07 11:11:39.762013 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:39.761289 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7ml6d" Jul 07 11:11:39.762013 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:39.761540 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2q46z" podUID="94c776b2-5ce9-4210-adf5-31efabcef2af" Jul 07 11:11:39.762013 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:39.761331 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sc5bl" Jul 07 11:11:39.762013 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:39.761609 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7ml6d" podUID="a4b1e5f7-71c6-4349-8dad-ef5763f9bfb9" Jul 07 11:11:39.762013 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:39.761660 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sc5bl" podUID="b9ca66a2-4c5a-4816-a081-bb0fecbdab5d" Jul 07 11:11:39.965097 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:39.965076 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tzf62_d2340b4b-179a-4c0a-9879-13b49897c939/ovn-acl-logging/0.log" Jul 07 11:11:39.965485 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:39.965453 2562 generic.go:356] "Generic (PLEG): container finished" podID="d2340b4b-179a-4c0a-9879-13b49897c939" containerID="6c392ef8ef208c799b66f2b91e84e3a8d3d4c68768e0b3c0cf045986b1d3b6fb" exitCode=1 Jul 07 11:11:39.965602 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:39.965515 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tzf62" event={"ID":"d2340b4b-179a-4c0a-9879-13b49897c939","Type":"ContainerStarted","Data":"f7a4c90c8a92f945fcb2bec32c17efb2f6f99923cd0a5e58d5ca31345f049fac"} Jul 07 11:11:39.965602 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:39.965596 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tzf62" event={"ID":"d2340b4b-179a-4c0a-9879-13b49897c939","Type":"ContainerStarted","Data":"dafa2c86a57addf1cf694e1fa43f5ef6502a1a483a592ae11fbc61ed5d34bc3a"} Jul 07 11:11:39.965714 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:39.965641 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tzf62" event={"ID":"d2340b4b-179a-4c0a-9879-13b49897c939","Type":"ContainerStarted","Data":"86f0d83eb2a710d16983972c5ae1c669d2130375c623801e0d458d798d1f9de8"} Jul 07 11:11:39.965714 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:39.965675 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tzf62" event={"ID":"d2340b4b-179a-4c0a-9879-13b49897c939","Type":"ContainerStarted","Data":"d7db954d1f638f320ce24b7d16fdea4638e4b8798bfabb9cc0635b370a617106"} Jul 07 11:11:39.965714 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:39.965708 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tzf62" event={"ID":"d2340b4b-179a-4c0a-9879-13b49897c939","Type":"ContainerDied","Data":"6c392ef8ef208c799b66f2b91e84e3a8d3d4c68768e0b3c0cf045986b1d3b6fb"} Jul 07 11:11:39.965842 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:39.965743 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tzf62" event={"ID":"d2340b4b-179a-4c0a-9879-13b49897c939","Type":"ContainerStarted","Data":"fcb763fba84f68be8b1f522e175fa0440a3b06a52c678fd136b10b043b224c3e"} Jul 07 11:11:39.967038 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:39.967014 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-trhvm" event={"ID":"95dba470-f153-4c3c-93d6-c3fb757fec0b","Type":"ContainerStarted","Data":"5ce13d6033793baadc748c4bfb31e497aca98fdf1a0c2e6057f86ca444915187"} Jul 07 11:11:39.968965 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:39.968943 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-vhcmx" event={"ID":"522a4c3d-a66a-4f5f-b8fa-e25cbf48c969","Type":"ContainerStarted","Data":"ebd3a7f279addddcf6a1f070a0406759e711942d99e9a9a15ad3ffe35fe809d2"} Jul 07 11:11:39.970399 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:39.970377 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-15.ec2.internal" event={"ID":"a0c235cfc97783f979ae1f6fa1e22e58","Type":"ContainerStarted","Data":"febd82820ecdbea1bfd68c029c6628cc11194ff09f5bf426b7edebe561bbaf95"} Jul 07 11:11:39.985223 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:39.985174 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-trhvm" podStartSLOduration=1.646289102 podStartE2EDuration="21.985158635s" totalImagesPullingTime="20.338869533s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 11:11:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:11:39.984945135 +0000 UTC m=+24.398814300" watchObservedRunningTime="2026-07-07 11:11:39.985158635 +0000 UTC m=+24.399027801" Jul 07 11:11:39.999740 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:39.999696 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-15.ec2.internal" podStartSLOduration=19.999679203 podStartE2EDuration="19.999679203s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 11:11:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:11:39.999362315 +0000 UTC m=+24.413231484" watchObservedRunningTime="2026-07-07 11:11:39.999679203 +0000 UTC m=+24.413548368" Jul 07 11:11:40.021625 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:40.021591 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-vhcmx" podStartSLOduration=1.9031434520000001 podStartE2EDuration="22.021578949s" totalImagesPullingTime="20.118435497s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 11:11:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:11:40.02141306 +0000 UTC m=+24.435282224" watchObservedRunningTime="2026-07-07 11:11:40.021578949 +0000 UTC m=+24.435448108" Jul 07 11:11:40.974411 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:40.974159 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-528n2" event={"ID":"e93d8868-ca25-4c0a-9f25-da7b8c286f60","Type":"ContainerStarted","Data":"6795e5150a97c02ad27b461826c26082bcd0f184545f76d53470b36389782987"} Jul 07 11:11:40.977979 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:40.977807 2562 generic.go:356] "Generic (PLEG): container finished" podID="6cc36837-424c-44fd-9644-1628e5e5d144" containerID="fbddd96dc0e4f3045e50850542e3a2a214ccae17d9855bfc8cfe631559f3b220" exitCode=0 Jul 07 11:11:40.977979 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:40.977899 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6h8jc" event={"ID":"6cc36837-424c-44fd-9644-1628e5e5d144","Type":"ContainerDied","Data":"fbddd96dc0e4f3045e50850542e3a2a214ccae17d9855bfc8cfe631559f3b220"} Jul 07 11:11:40.980099 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:40.980069 2562 generic.go:356] "Generic (PLEG): container finished" podID="dcf962a5f27f6d615415b27111cc76dc" containerID="bfbff7e2a8227a366787e0dd0151e268dedb2e7e6d90913b71a03bd7dda3fef5" exitCode=0 Jul 07 11:11:40.980200 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:40.980127 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-15.ec2.internal" event={"ID":"dcf962a5f27f6d615415b27111cc76dc","Type":"ContainerDied","Data":"bfbff7e2a8227a366787e0dd0151e268dedb2e7e6d90913b71a03bd7dda3fef5"} Jul 07 11:11:40.981892 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:40.981869 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-dvhgx" event={"ID":"d4ebce69-4659-439d-b4fa-8e9f4b17ec2a","Type":"ContainerStarted","Data":"9be880dd72e6837db6992573de5006c30d317b49e841218b6d9194ae7d82069b"} Jul 07 11:11:40.983610 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:40.983581 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-4b5qw" event={"ID":"663d4847-eafe-43d6-a4b8-2bec15919119","Type":"ContainerStarted","Data":"6a540090d2090522139bd5b6b85aadb69753216e09eeced9cc7be02a4fcd7592"} Jul 07 11:11:40.985268 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:40.985239 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n8tzm" event={"ID":"5e4c9eaf-157e-4ada-8ee8-3d9319eb26a9","Type":"ContainerStarted","Data":"28b84da8270c887e11cdcdb02c0eeb9b36adbecaa4e54f30ce63b8eb83f43ae9"} Jul 07 11:11:40.986857 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:40.986831 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-77v22" event={"ID":"a9da5840-e6b4-4719-98cd-ef0d46d8470c","Type":"ContainerStarted","Data":"d4ffdfee93aac1ba2b471b37b46951a0f9cc9cf2298492f615ed8dcb3d774ef1"} Jul 07 11:11:40.989647 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:40.989611 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-528n2" podStartSLOduration=2.8698771 podStartE2EDuration="22.989598904s" totalImagesPullingTime="20.119721804s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 11:11:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:11:40.989448846 +0000 UTC m=+25.403318011" watchObservedRunningTime="2026-07-07 11:11:40.989598904 +0000 UTC m=+25.403468069" Jul 07 11:11:41.005164 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:41.005128 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-4b5qw" podStartSLOduration=2.967829223 podStartE2EDuration="23.005115528s" totalImagesPullingTime="20.037286305s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:11:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:11:41.004258201 +0000 UTC m=+25.418127368" watchObservedRunningTime="2026-07-07 11:11:41.005115528 +0000 UTC m=+25.418984694" Jul 07 11:11:41.025069 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:41.023525 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-77v22" podStartSLOduration=2.92656179 podStartE2EDuration="23.023508664s" totalImagesPullingTime="20.096946874s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 11:11:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:11:41.022722592 +0000 UTC m=+25.436591756" watchObservedRunningTime="2026-07-07 11:11:41.023508664 +0000 UTC m=+25.437377829" Jul 07 11:11:41.060213 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:41.060188 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:11:41.164381 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:41.164356 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:11:41.262407 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:41.262352 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:11:41.377173 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:41.377142 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:11:41.460193 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:41.460174 2562 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 07 11:11:41.483534 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:41.483512 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:11:41.658513 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:41.658439 2562 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-07T11:11:41.460188712Z","UUID":"64199bd6-5466-4242-ad71-fe0da33c84ef","Handler":null,"Name":"","Endpoint":""} Jul 07 11:11:41.660044 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:41.660024 2562 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 07 11:11:41.660177 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:41.660063 2562 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 07 11:11:41.761404 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:41.761319 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7ml6d" Jul 07 11:11:41.761404 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:41.761360 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2q46z" Jul 07 11:11:41.761626 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:41.761439 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7ml6d" podUID="a4b1e5f7-71c6-4349-8dad-ef5763f9bfb9" Jul 07 11:11:41.761626 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:41.761529 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2q46z" podUID="94c776b2-5ce9-4210-adf5-31efabcef2af" Jul 07 11:11:41.761626 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:41.761587 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sc5bl" Jul 07 11:11:41.761786 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:41.761652 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sc5bl" podUID="b9ca66a2-4c5a-4816-a081-bb0fecbdab5d" Jul 07 11:11:41.991059 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:41.990980 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-15.ec2.internal" event={"ID":"dcf962a5f27f6d615415b27111cc76dc","Type":"ContainerStarted","Data":"5cf5f173a193007289fceeae55e02d8c87527dea5c4e56421c1f48e5c63b356b"} Jul 07 11:11:41.993212 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:41.993160 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n8tzm" event={"ID":"5e4c9eaf-157e-4ada-8ee8-3d9319eb26a9","Type":"ContainerStarted","Data":"560cb7648a3049add9f960b59328fb1bd44564e4a0b23b0b4b292609a898a1cd"} Jul 07 11:11:41.997596 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:41.997576 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tzf62_d2340b4b-179a-4c0a-9879-13b49897c939/ovn-acl-logging/0.log" Jul 07 11:11:41.998000 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:41.997976 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tzf62" event={"ID":"d2340b4b-179a-4c0a-9879-13b49897c939","Type":"ContainerStarted","Data":"d890f33711a24b1bb369fc073894fc6d0e1e9ce9477c6667b84357f6c4dfc993"} Jul 07 11:11:42.011323 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:42.011278 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-dvhgx" podStartSLOduration=24.01125978 podStartE2EDuration="24.01125978s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 11:11:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=1 observedRunningTime="2026-07-07 11:11:41.075095451 +0000 UTC m=+25.488964617" watchObservedRunningTime="2026-07-07 11:11:42.01125978 +0000 UTC m=+26.425128946" Jul 07 11:11:42.011441 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:42.011380 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-15.ec2.internal" podStartSLOduration=22.011372512 podStartE2EDuration="22.011372512s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 11:11:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:11:42.011178437 +0000 UTC m=+26.425047603" watchObservedRunningTime="2026-07-07 11:11:42.011372512 +0000 UTC m=+26.425241681" Jul 07 11:11:42.324563 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:42.324425 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:11:42.660297 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:42.660273 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:11:43.002248 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:43.002215 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n8tzm" event={"ID":"5e4c9eaf-157e-4ada-8ee8-3d9319eb26a9","Type":"ContainerStarted","Data":"0299dc0139514a84217959abf7e0ca723b6d7c87adbddf5cc6539a4024982f28"} Jul 07 11:11:43.022561 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:43.022521 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n8tzm" podStartSLOduration=2.545084849 podStartE2EDuration="25.022508529s" totalImagesPullingTime="22.47742368s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 11:11:18 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:11:43.022399568 +0000 UTC m=+27.436268732" watchObservedRunningTime="2026-07-07 11:11:43.022508529 +0000 UTC m=+27.436377693" Jul 07 11:11:43.621901 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:43.621872 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a4b1e5f7-71c6-4349-8dad-ef5763f9bfb9-original-pull-secret\") pod \"global-pull-secret-syncer-7ml6d\" (UID: \"a4b1e5f7-71c6-4349-8dad-ef5763f9bfb9\") " pod="kube-system/global-pull-secret-syncer-7ml6d" Jul 07 11:11:43.622084 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:43.622040 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 11:11:43.622149 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:43.622097 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a4b1e5f7-71c6-4349-8dad-ef5763f9bfb9-original-pull-secret podName:a4b1e5f7-71c6-4349-8dad-ef5763f9bfb9 nodeName:}" failed. No retries permitted until 2026-07-07 11:11:59.622082095 +0000 UTC m=+44.035951238 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a4b1e5f7-71c6-4349-8dad-ef5763f9bfb9-original-pull-secret") pod "global-pull-secret-syncer-7ml6d" (UID: "a4b1e5f7-71c6-4349-8dad-ef5763f9bfb9") : object "kube-system"/"original-pull-secret" not registered Jul 07 11:11:43.761181 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:43.761153 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7ml6d" Jul 07 11:11:43.761181 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:43.761180 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2q46z" Jul 07 11:11:43.761372 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:43.761277 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7ml6d" podUID="a4b1e5f7-71c6-4349-8dad-ef5763f9bfb9" Jul 07 11:11:43.761588 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:43.761546 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sc5bl" Jul 07 11:11:43.761708 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:43.761635 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2q46z" podUID="94c776b2-5ce9-4210-adf5-31efabcef2af" Jul 07 11:11:43.761777 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:43.761738 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sc5bl" podUID="b9ca66a2-4c5a-4816-a081-bb0fecbdab5d" Jul 07 11:11:44.008675 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:44.008570 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tzf62_d2340b4b-179a-4c0a-9879-13b49897c939/ovn-acl-logging/0.log" Jul 07 11:11:44.009147 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:44.009022 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tzf62" event={"ID":"d2340b4b-179a-4c0a-9879-13b49897c939","Type":"ContainerStarted","Data":"1f108a4b569ed4555947535d9494285de0c4d3c4e04ec5dae28ea1489dc5978c"} Jul 07 11:11:44.009580 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:44.009561 2562 scope.go:117] "RemoveContainer" containerID="6c392ef8ef208c799b66f2b91e84e3a8d3d4c68768e0b3c0cf045986b1d3b6fb" Jul 07 11:11:45.015449 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:45.015423 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tzf62_d2340b4b-179a-4c0a-9879-13b49897c939/ovn-acl-logging/0.log" Jul 07 11:11:45.015997 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:45.015817 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tzf62" event={"ID":"d2340b4b-179a-4c0a-9879-13b49897c939","Type":"ContainerStarted","Data":"44aa6c0d47261119e7e6b3b007eaa65fffec832a89db15ecfa1c9a72060881c8"} Jul 07 11:11:45.016154 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:45.016134 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-tzf62" Jul 07 11:11:45.016236 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:45.016213 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-tzf62" Jul 07 11:11:45.016277 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:45.016261 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-tzf62" Jul 07 11:11:45.019896 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:45.019413 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-4b5qw" Jul 07 11:11:45.020231 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:45.020213 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-4b5qw" Jul 07 11:11:45.034428 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:45.034407 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-tzf62" Jul 07 11:11:45.034730 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:45.034707 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-tzf62" Jul 07 11:11:45.050589 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:45.050553 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-tzf62" podStartSLOduration=6.867304986 podStartE2EDuration="27.050540728s" totalImagesPullingTime="20.183235742s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 11:11:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:11:45.0490852 +0000 UTC m=+29.462954376" watchObservedRunningTime="2026-07-07 11:11:45.050540728 +0000 UTC m=+29.464409892" Jul 07 11:11:45.666193 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:45.666148 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:11:45.761477 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:45.761348 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7ml6d" Jul 07 11:11:45.761477 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:45.761359 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sc5bl" Jul 07 11:11:45.761477 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:45.761469 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7ml6d" podUID="a4b1e5f7-71c6-4349-8dad-ef5763f9bfb9" Jul 07 11:11:45.761767 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:45.761359 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2q46z" Jul 07 11:11:45.761767 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:45.761527 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sc5bl" podUID="b9ca66a2-4c5a-4816-a081-bb0fecbdab5d" Jul 07 11:11:45.761767 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:45.761590 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2q46z" podUID="94c776b2-5ce9-4210-adf5-31efabcef2af" Jul 07 11:11:46.018742 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:46.018679 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-4b5qw" Jul 07 11:11:46.019150 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:46.018751 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-4b5qw" Jul 07 11:11:46.481293 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:46.481252 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:11:46.839934 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:46.839846 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:11:46.934435 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:46.934406 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:11:47.093716 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:47.093654 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:11:47.380554 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:47.380356 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:11:47.762242 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:47.762164 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7ml6d" Jul 07 11:11:47.762370 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:47.762257 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sc5bl" Jul 07 11:11:47.762370 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:47.762277 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7ml6d" podUID="a4b1e5f7-71c6-4349-8dad-ef5763f9bfb9" Jul 07 11:11:47.762370 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:47.762308 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2q46z" Jul 07 11:11:47.762465 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:47.762379 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2q46z" podUID="94c776b2-5ce9-4210-adf5-31efabcef2af" Jul 07 11:11:47.762465 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:47.762440 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sc5bl" podUID="b9ca66a2-4c5a-4816-a081-bb0fecbdab5d" Jul 07 11:11:47.771112 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:47.771086 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:11:48.023798 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:48.023720 2562 generic.go:356] "Generic (PLEG): container finished" podID="6cc36837-424c-44fd-9644-1628e5e5d144" containerID="d9457357abe3717e604ca489b5f75e33531190e8228bf2575a0999d75ea1b968" exitCode=0 Jul 07 11:11:48.023798 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:48.023759 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6h8jc" event={"ID":"6cc36837-424c-44fd-9644-1628e5e5d144","Type":"ContainerDied","Data":"d9457357abe3717e604ca489b5f75e33531190e8228bf2575a0999d75ea1b968"} Jul 07 11:11:48.374972 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:48.374943 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:11:48.764649 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:48.764561 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:11:49.171132 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:49.171094 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:11:49.563349 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:49.563321 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:11:49.761206 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:49.761182 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7ml6d" Jul 07 11:11:49.761206 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:49.761176 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sc5bl" Jul 07 11:11:49.761347 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:49.761191 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2q46z" Jul 07 11:11:49.761347 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:49.761276 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7ml6d" podUID="a4b1e5f7-71c6-4349-8dad-ef5763f9bfb9" Jul 07 11:11:49.761406 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:49.761345 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2q46z" podUID="94c776b2-5ce9-4210-adf5-31efabcef2af" Jul 07 11:11:49.761440 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:49.761418 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sc5bl" podUID="b9ca66a2-4c5a-4816-a081-bb0fecbdab5d" Jul 07 11:11:49.768803 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:49.768778 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/94c776b2-5ce9-4210-adf5-31efabcef2af-metrics-certs\") pod \"network-metrics-daemon-2q46z\" (UID: \"94c776b2-5ce9-4210-adf5-31efabcef2af\") " pod="openshift-multus/network-metrics-daemon-2q46z" Jul 07 11:11:49.768939 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:49.768909 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 11:11:49.768988 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:49.768974 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/94c776b2-5ce9-4210-adf5-31efabcef2af-metrics-certs podName:94c776b2-5ce9-4210-adf5-31efabcef2af nodeName:}" failed. No retries permitted until 2026-07-07 11:12:21.768961997 +0000 UTC m=+66.182831140 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/94c776b2-5ce9-4210-adf5-31efabcef2af-metrics-certs") pod "network-metrics-daemon-2q46z" (UID: "94c776b2-5ce9-4210-adf5-31efabcef2af") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 11:11:49.771532 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:49.771512 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:11:49.970448 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:49.970413 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7nvg6\" (UniqueName: \"kubernetes.io/projected/b9ca66a2-4c5a-4816-a081-bb0fecbdab5d-kube-api-access-7nvg6\") pod \"network-check-target-sc5bl\" (UID: \"b9ca66a2-4c5a-4816-a081-bb0fecbdab5d\") " pod="openshift-network-diagnostics/network-check-target-sc5bl" Jul 07 11:11:49.970575 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:49.970558 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 11:11:49.970627 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:49.970587 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 11:11:49.970627 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:49.970600 2562 projected.go:196] Error preparing data for projected volume kube-api-access-7nvg6 for pod openshift-network-diagnostics/network-check-target-sc5bl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 11:11:49.970687 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:49.970645 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b9ca66a2-4c5a-4816-a081-bb0fecbdab5d-kube-api-access-7nvg6 podName:b9ca66a2-4c5a-4816-a081-bb0fecbdab5d nodeName:}" failed. No retries permitted until 2026-07-07 11:12:21.970631815 +0000 UTC m=+66.384500959 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-7nvg6" (UniqueName: "kubernetes.io/projected/b9ca66a2-4c5a-4816-a081-bb0fecbdab5d-kube-api-access-7nvg6") pod "network-check-target-sc5bl" (UID: "b9ca66a2-4c5a-4816-a081-bb0fecbdab5d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 11:11:50.031123 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:50.031095 2562 generic.go:356] "Generic (PLEG): container finished" podID="6cc36837-424c-44fd-9644-1628e5e5d144" containerID="530d3b9d435686d3f8c11f65e7ac4f9e388a9c307b0dbabc95a7211486efb488" exitCode=0 Jul 07 11:11:50.031209 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:50.031136 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6h8jc" event={"ID":"6cc36837-424c-44fd-9644-1628e5e5d144","Type":"ContainerDied","Data":"530d3b9d435686d3f8c11f65e7ac4f9e388a9c307b0dbabc95a7211486efb488"} Jul 07 11:11:50.179001 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:50.178977 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:11:50.324244 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:50.324183 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:11:50.443978 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:50.443954 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:11:50.783760 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:50.783732 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:11:51.169459 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:51.169436 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:11:51.243634 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:51.243610 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:11:51.567994 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:51.567966 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:11:51.761469 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:51.761444 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sc5bl" Jul 07 11:11:51.761611 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:51.761445 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2q46z" Jul 07 11:11:51.761611 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:51.761575 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7ml6d" Jul 07 11:11:51.761709 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:51.761546 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sc5bl" podUID="b9ca66a2-4c5a-4816-a081-bb0fecbdab5d" Jul 07 11:11:51.761709 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:51.761648 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2q46z" podUID="94c776b2-5ce9-4210-adf5-31efabcef2af" Jul 07 11:11:51.761801 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:51.761711 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7ml6d" podUID="a4b1e5f7-71c6-4349-8dad-ef5763f9bfb9" Jul 07 11:11:51.765435 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:51.765403 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:11:51.971389 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:51.971364 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:11:52.037545 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:52.037519 2562 generic.go:356] "Generic (PLEG): container finished" podID="6cc36837-424c-44fd-9644-1628e5e5d144" containerID="03ee0221211c6b1cf78bee9f9729ea821d22a22fed195c241962635bd5148f0a" exitCode=0 Jul 07 11:11:52.037656 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:52.037563 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6h8jc" event={"ID":"6cc36837-424c-44fd-9644-1628e5e5d144","Type":"ContainerDied","Data":"03ee0221211c6b1cf78bee9f9729ea821d22a22fed195c241962635bd5148f0a"} Jul 07 11:11:52.699269 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:52.699239 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:11:52.818122 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:52.818092 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:11:53.163738 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:53.163713 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:11:53.374120 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:53.374084 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:11:53.665454 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:53.665425 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:11:53.760691 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:53.760664 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7ml6d" Jul 07 11:11:53.760885 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:53.760664 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sc5bl" Jul 07 11:11:53.760885 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:53.760758 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7ml6d" podUID="a4b1e5f7-71c6-4349-8dad-ef5763f9bfb9" Jul 07 11:11:53.760885 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:53.760677 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2q46z" Jul 07 11:11:53.760885 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:53.760843 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sc5bl" podUID="b9ca66a2-4c5a-4816-a081-bb0fecbdab5d" Jul 07 11:11:53.761043 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:53.760975 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2q46z" podUID="94c776b2-5ce9-4210-adf5-31efabcef2af" Jul 07 11:11:53.964157 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:53.964050 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:11:54.368727 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:54.368654 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:11:54.764751 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:54.764711 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:11:54.968123 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:54.968091 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:11:55.570564 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:55.570529 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:11:55.761333 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:55.761300 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2q46z" Jul 07 11:11:55.761333 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:55.761333 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7ml6d" Jul 07 11:11:55.761528 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:55.761300 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sc5bl" Jul 07 11:11:55.761528 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:55.761437 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7ml6d" podUID="a4b1e5f7-71c6-4349-8dad-ef5763f9bfb9" Jul 07 11:11:55.761528 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:55.761517 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2q46z" podUID="94c776b2-5ce9-4210-adf5-31efabcef2af" Jul 07 11:11:55.761634 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:55.761601 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sc5bl" podUID="b9ca66a2-4c5a-4816-a081-bb0fecbdab5d" Jul 07 11:11:55.966534 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:55.966507 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:11:56.372892 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:56.372807 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:11:56.598574 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:56.598537 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:11:57.005231 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:57.005196 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:11:57.171550 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:57.171514 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:11:57.556470 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:57.556435 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:11:57.658074 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:57.658048 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:11:57.762073 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:57.762046 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2q46z" Jul 07 11:11:57.762176 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:57.762137 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2q46z" podUID="94c776b2-5ce9-4210-adf5-31efabcef2af" Jul 07 11:11:57.762176 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:57.762147 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sc5bl" Jul 07 11:11:57.762293 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:57.762215 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7ml6d" Jul 07 11:11:57.762344 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:57.762314 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7ml6d" podUID="a4b1e5f7-71c6-4349-8dad-ef5763f9bfb9" Jul 07 11:11:57.762478 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:57.762447 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sc5bl" podUID="b9ca66a2-4c5a-4816-a081-bb0fecbdab5d" Jul 07 11:11:58.053889 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:58.053858 2562 generic.go:356] "Generic (PLEG): container finished" podID="6cc36837-424c-44fd-9644-1628e5e5d144" containerID="e1fa76cbc60b4ef1433d25727fdf74e3a69e435b802b173e85fa5358825d3da5" exitCode=0 Jul 07 11:11:58.054018 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:58.053900 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6h8jc" event={"ID":"6cc36837-424c-44fd-9644-1628e5e5d144","Type":"ContainerDied","Data":"e1fa76cbc60b4ef1433d25727fdf74e3a69e435b802b173e85fa5358825d3da5"} Jul 07 11:11:59.059009 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:59.058976 2562 generic.go:356] "Generic (PLEG): container finished" podID="6cc36837-424c-44fd-9644-1628e5e5d144" containerID="b0e614332ff40c859289aed49cef459c62b35f9d950385e8679346650285674e" exitCode=0 Jul 07 11:11:59.059373 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:59.059033 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6h8jc" event={"ID":"6cc36837-424c-44fd-9644-1628e5e5d144","Type":"ContainerDied","Data":"b0e614332ff40c859289aed49cef459c62b35f9d950385e8679346650285674e"} Jul 07 11:11:59.643070 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:59.643035 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a4b1e5f7-71c6-4349-8dad-ef5763f9bfb9-original-pull-secret\") pod \"global-pull-secret-syncer-7ml6d\" (UID: \"a4b1e5f7-71c6-4349-8dad-ef5763f9bfb9\") " pod="kube-system/global-pull-secret-syncer-7ml6d" Jul 07 11:11:59.643190 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:59.643170 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 11:11:59.643239 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:59.643225 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a4b1e5f7-71c6-4349-8dad-ef5763f9bfb9-original-pull-secret podName:a4b1e5f7-71c6-4349-8dad-ef5763f9bfb9 nodeName:}" failed. No retries permitted until 2026-07-07 11:12:31.643211118 +0000 UTC m=+76.057080261 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a4b1e5f7-71c6-4349-8dad-ef5763f9bfb9-original-pull-secret") pod "global-pull-secret-syncer-7ml6d" (UID: "a4b1e5f7-71c6-4349-8dad-ef5763f9bfb9") : object "kube-system"/"original-pull-secret" not registered Jul 07 11:11:59.761583 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:59.761551 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2q46z" Jul 07 11:11:59.761706 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:59.761551 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sc5bl" Jul 07 11:11:59.761706 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:59.761669 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2q46z" podUID="94c776b2-5ce9-4210-adf5-31efabcef2af" Jul 07 11:11:59.761820 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:59.761726 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sc5bl" podUID="b9ca66a2-4c5a-4816-a081-bb0fecbdab5d" Jul 07 11:11:59.761820 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:11:59.761772 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7ml6d" Jul 07 11:11:59.761948 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:11:59.761845 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7ml6d" podUID="a4b1e5f7-71c6-4349-8dad-ef5763f9bfb9" Jul 07 11:12:00.064964 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:00.064881 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6h8jc" event={"ID":"6cc36837-424c-44fd-9644-1628e5e5d144","Type":"ContainerStarted","Data":"d9f92203e766e14225d0407e41176971abbaade768a841e79c57d6f1e993b6ef"} Jul 07 11:12:00.091271 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:00.091220 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-6h8jc" podStartSLOduration=8.300580739 podStartE2EDuration="42.091207225s" totalImagesPullingTime="33.790626486s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 11:11:18 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:12:00.090642757 +0000 UTC m=+44.504511922" watchObservedRunningTime="2026-07-07 11:12:00.091207225 +0000 UTC m=+44.505076390" Jul 07 11:12:01.761168 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:01.761130 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sc5bl" Jul 07 11:12:01.761168 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:01.761154 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2q46z" Jul 07 11:12:01.761717 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:01.761138 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7ml6d" Jul 07 11:12:01.761717 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:12:01.761241 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sc5bl" podUID="b9ca66a2-4c5a-4816-a081-bb0fecbdab5d" Jul 07 11:12:01.761717 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:12:01.761315 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7ml6d" podUID="a4b1e5f7-71c6-4349-8dad-ef5763f9bfb9" Jul 07 11:12:01.761717 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:12:01.761403 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2q46z" podUID="94c776b2-5ce9-4210-adf5-31efabcef2af" Jul 07 11:12:03.761139 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:03.761103 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7ml6d" Jul 07 11:12:03.761661 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:03.761109 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2q46z" Jul 07 11:12:03.761661 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:03.761214 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sc5bl" Jul 07 11:12:03.761661 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:12:03.761206 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7ml6d" podUID="a4b1e5f7-71c6-4349-8dad-ef5763f9bfb9" Jul 07 11:12:03.761661 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:12:03.761295 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sc5bl" podUID="b9ca66a2-4c5a-4816-a081-bb0fecbdab5d" Jul 07 11:12:03.761661 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:12:03.761389 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2q46z" podUID="94c776b2-5ce9-4210-adf5-31efabcef2af" Jul 07 11:12:04.179272 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:04.179222 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:12:04.278881 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:04.278853 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:12:04.382534 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:04.382505 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:12:04.480532 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:04.480477 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:12:04.582462 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:04.582439 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:12:05.306726 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:05.306509 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-7ml6d"] Jul 07 11:12:05.307098 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:05.306820 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7ml6d" Jul 07 11:12:05.307098 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:12:05.306953 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7ml6d" podUID="a4b1e5f7-71c6-4349-8dad-ef5763f9bfb9" Jul 07 11:12:05.309492 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:05.309470 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-sc5bl"] Jul 07 11:12:05.309633 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:05.309554 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sc5bl" Jul 07 11:12:05.309699 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:12:05.309645 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sc5bl" podUID="b9ca66a2-4c5a-4816-a081-bb0fecbdab5d" Jul 07 11:12:05.310318 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:05.310294 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-2q46z"] Jul 07 11:12:05.310429 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:05.310415 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2q46z" Jul 07 11:12:05.310543 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:12:05.310523 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2q46z" podUID="94c776b2-5ce9-4210-adf5-31efabcef2af" Jul 07 11:12:05.476369 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:05.476342 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:12:05.710175 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:05.710141 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:12:05.780065 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:05.780037 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:12:06.761244 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:06.761216 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sc5bl" Jul 07 11:12:06.761599 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:06.761305 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2q46z" Jul 07 11:12:06.761599 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:12:06.761312 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sc5bl" podUID="b9ca66a2-4c5a-4816-a081-bb0fecbdab5d" Jul 07 11:12:06.761599 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:12:06.761397 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2q46z" podUID="94c776b2-5ce9-4210-adf5-31efabcef2af" Jul 07 11:12:06.761599 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:06.761435 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7ml6d" Jul 07 11:12:06.761599 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:12:06.761495 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7ml6d" podUID="a4b1e5f7-71c6-4349-8dad-ef5763f9bfb9" Jul 07 11:12:07.155259 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:07.155229 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:12:07.252312 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:07.252285 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:12:07.330742 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:07.330714 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:12:07.522233 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:07.522159 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:12:08.761388 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:08.761355 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sc5bl" Jul 07 11:12:08.761749 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:08.761457 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7ml6d" Jul 07 11:12:08.761749 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:12:08.761467 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sc5bl" podUID="b9ca66a2-4c5a-4816-a081-bb0fecbdab5d" Jul 07 11:12:08.761749 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:08.761481 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2q46z" Jul 07 11:12:08.761749 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:12:08.761541 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7ml6d" podUID="a4b1e5f7-71c6-4349-8dad-ef5763f9bfb9" Jul 07 11:12:08.761749 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:12:08.761670 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2q46z" podUID="94c776b2-5ce9-4210-adf5-31efabcef2af" Jul 07 11:12:09.889453 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:09.889425 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-15.ec2.internal" event="NodeReady" Jul 07 11:12:09.889850 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:09.889520 2562 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 07 11:12:09.959164 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:09.959099 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-k8wv9"] Jul 07 11:12:09.976735 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:09.976709 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-mmtwk"] Jul 07 11:12:09.976857 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:09.976818 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-k8wv9" Jul 07 11:12:09.980116 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:09.980094 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 07 11:12:09.980778 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:09.980763 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-2gjl2\"" Jul 07 11:12:09.981040 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:09.981023 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 07 11:12:09.997453 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:09.997435 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-mmtwk"] Jul 07 11:12:09.997538 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:09.997462 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-k8wv9"] Jul 07 11:12:09.997538 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:09.997530 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-mmtwk" Jul 07 11:12:10.001443 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:10.001348 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 07 11:12:10.002051 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:10.002030 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 07 11:12:10.002150 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:10.002084 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-jjlcq\"" Jul 07 11:12:10.002150 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:10.002081 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 07 11:12:10.023808 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:10.023780 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c5da13ef-838f-4deb-9b49-7b12cab2bb84-config-volume\") pod \"dns-default-k8wv9\" (UID: \"c5da13ef-838f-4deb-9b49-7b12cab2bb84\") " pod="openshift-dns/dns-default-k8wv9" Jul 07 11:12:10.023910 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:10.023834 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gs9dc\" (UniqueName: \"kubernetes.io/projected/0bcfaddb-0c6d-4519-a1a7-c23b3a610272-kube-api-access-gs9dc\") pod \"ingress-canary-mmtwk\" (UID: \"0bcfaddb-0c6d-4519-a1a7-c23b3a610272\") " pod="openshift-ingress-canary/ingress-canary-mmtwk" Jul 07 11:12:10.023910 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:10.023872 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f5mwg\" (UniqueName: \"kubernetes.io/projected/c5da13ef-838f-4deb-9b49-7b12cab2bb84-kube-api-access-f5mwg\") pod \"dns-default-k8wv9\" (UID: \"c5da13ef-838f-4deb-9b49-7b12cab2bb84\") " pod="openshift-dns/dns-default-k8wv9" Jul 07 11:12:10.023910 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:10.023894 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c5da13ef-838f-4deb-9b49-7b12cab2bb84-metrics-tls\") pod \"dns-default-k8wv9\" (UID: \"c5da13ef-838f-4deb-9b49-7b12cab2bb84\") " pod="openshift-dns/dns-default-k8wv9" Jul 07 11:12:10.024094 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:10.023941 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0bcfaddb-0c6d-4519-a1a7-c23b3a610272-cert\") pod \"ingress-canary-mmtwk\" (UID: \"0bcfaddb-0c6d-4519-a1a7-c23b3a610272\") " pod="openshift-ingress-canary/ingress-canary-mmtwk" Jul 07 11:12:10.024094 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:10.024019 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c5da13ef-838f-4deb-9b49-7b12cab2bb84-tmp-dir\") pod \"dns-default-k8wv9\" (UID: \"c5da13ef-838f-4deb-9b49-7b12cab2bb84\") " pod="openshift-dns/dns-default-k8wv9" Jul 07 11:12:10.055002 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:10.054977 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-5279v"] Jul 07 11:12:10.072278 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:10.072255 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-5279v" Jul 07 11:12:10.080384 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:10.080366 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 07 11:12:10.080557 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:10.080541 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-tnjrv\"" Jul 07 11:12:10.080869 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:10.080854 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 07 11:12:10.081332 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:10.081318 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 07 11:12:10.082291 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:10.082278 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 07 11:12:10.091056 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:10.091037 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-5279v"] Jul 07 11:12:10.124646 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:10.124625 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c5da13ef-838f-4deb-9b49-7b12cab2bb84-config-volume\") pod \"dns-default-k8wv9\" (UID: \"c5da13ef-838f-4deb-9b49-7b12cab2bb84\") " pod="openshift-dns/dns-default-k8wv9" Jul 07 11:12:10.124785 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:10.124676 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gs9dc\" (UniqueName: \"kubernetes.io/projected/0bcfaddb-0c6d-4519-a1a7-c23b3a610272-kube-api-access-gs9dc\") pod \"ingress-canary-mmtwk\" (UID: \"0bcfaddb-0c6d-4519-a1a7-c23b3a610272\") " pod="openshift-ingress-canary/ingress-canary-mmtwk" Jul 07 11:12:10.124785 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:10.124709 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f5mwg\" (UniqueName: \"kubernetes.io/projected/c5da13ef-838f-4deb-9b49-7b12cab2bb84-kube-api-access-f5mwg\") pod \"dns-default-k8wv9\" (UID: \"c5da13ef-838f-4deb-9b49-7b12cab2bb84\") " pod="openshift-dns/dns-default-k8wv9" Jul 07 11:12:10.124785 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:10.124727 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/267a6b3c-8130-497b-b850-fca340df911a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5279v\" (UID: \"267a6b3c-8130-497b-b850-fca340df911a\") " pod="openshift-insights/insights-runtime-extractor-5279v" Jul 07 11:12:10.124785 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:10.124745 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c5da13ef-838f-4deb-9b49-7b12cab2bb84-metrics-tls\") pod \"dns-default-k8wv9\" (UID: \"c5da13ef-838f-4deb-9b49-7b12cab2bb84\") " pod="openshift-dns/dns-default-k8wv9" Jul 07 11:12:10.124785 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:10.124773 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/267a6b3c-8130-497b-b850-fca340df911a-crio-socket\") pod \"insights-runtime-extractor-5279v\" (UID: \"267a6b3c-8130-497b-b850-fca340df911a\") " pod="openshift-insights/insights-runtime-extractor-5279v" Jul 07 11:12:10.125071 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:10.124789 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/267a6b3c-8130-497b-b850-fca340df911a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5279v\" (UID: \"267a6b3c-8130-497b-b850-fca340df911a\") " pod="openshift-insights/insights-runtime-extractor-5279v" Jul 07 11:12:10.125071 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:10.124810 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tt6v5\" (UniqueName: \"kubernetes.io/projected/267a6b3c-8130-497b-b850-fca340df911a-kube-api-access-tt6v5\") pod \"insights-runtime-extractor-5279v\" (UID: \"267a6b3c-8130-497b-b850-fca340df911a\") " pod="openshift-insights/insights-runtime-extractor-5279v" Jul 07 11:12:10.125071 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:10.124872 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0bcfaddb-0c6d-4519-a1a7-c23b3a610272-cert\") pod \"ingress-canary-mmtwk\" (UID: \"0bcfaddb-0c6d-4519-a1a7-c23b3a610272\") " pod="openshift-ingress-canary/ingress-canary-mmtwk" Jul 07 11:12:10.125071 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:10.124955 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c5da13ef-838f-4deb-9b49-7b12cab2bb84-tmp-dir\") pod \"dns-default-k8wv9\" (UID: \"c5da13ef-838f-4deb-9b49-7b12cab2bb84\") " pod="openshift-dns/dns-default-k8wv9" Jul 07 11:12:10.125071 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:10.124990 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/267a6b3c-8130-497b-b850-fca340df911a-data-volume\") pod \"insights-runtime-extractor-5279v\" (UID: \"267a6b3c-8130-497b-b850-fca340df911a\") " pod="openshift-insights/insights-runtime-extractor-5279v" Jul 07 11:12:10.125271 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:10.125238 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c5da13ef-838f-4deb-9b49-7b12cab2bb84-config-volume\") pod \"dns-default-k8wv9\" (UID: \"c5da13ef-838f-4deb-9b49-7b12cab2bb84\") " pod="openshift-dns/dns-default-k8wv9" Jul 07 11:12:10.125271 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:10.125236 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c5da13ef-838f-4deb-9b49-7b12cab2bb84-tmp-dir\") pod \"dns-default-k8wv9\" (UID: \"c5da13ef-838f-4deb-9b49-7b12cab2bb84\") " pod="openshift-dns/dns-default-k8wv9" Jul 07 11:12:10.128550 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:10.128531 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0bcfaddb-0c6d-4519-a1a7-c23b3a610272-cert\") pod \"ingress-canary-mmtwk\" (UID: \"0bcfaddb-0c6d-4519-a1a7-c23b3a610272\") " pod="openshift-ingress-canary/ingress-canary-mmtwk" Jul 07 11:12:10.128638 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:10.128529 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c5da13ef-838f-4deb-9b49-7b12cab2bb84-metrics-tls\") pod \"dns-default-k8wv9\" (UID: \"c5da13ef-838f-4deb-9b49-7b12cab2bb84\") " pod="openshift-dns/dns-default-k8wv9" Jul 07 11:12:10.139860 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:10.139835 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f5mwg\" (UniqueName: \"kubernetes.io/projected/c5da13ef-838f-4deb-9b49-7b12cab2bb84-kube-api-access-f5mwg\") pod \"dns-default-k8wv9\" (UID: \"c5da13ef-838f-4deb-9b49-7b12cab2bb84\") " pod="openshift-dns/dns-default-k8wv9" Jul 07 11:12:10.139965 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:10.139905 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gs9dc\" (UniqueName: \"kubernetes.io/projected/0bcfaddb-0c6d-4519-a1a7-c23b3a610272-kube-api-access-gs9dc\") pod \"ingress-canary-mmtwk\" (UID: \"0bcfaddb-0c6d-4519-a1a7-c23b3a610272\") " pod="openshift-ingress-canary/ingress-canary-mmtwk" Jul 07 11:12:10.179575 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:10.179551 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:12:10.225761 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:10.225705 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/267a6b3c-8130-497b-b850-fca340df911a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5279v\" (UID: \"267a6b3c-8130-497b-b850-fca340df911a\") " pod="openshift-insights/insights-runtime-extractor-5279v" Jul 07 11:12:10.225761 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:10.225746 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/267a6b3c-8130-497b-b850-fca340df911a-crio-socket\") pod \"insights-runtime-extractor-5279v\" (UID: \"267a6b3c-8130-497b-b850-fca340df911a\") " pod="openshift-insights/insights-runtime-extractor-5279v" Jul 07 11:12:10.225932 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:10.225773 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/267a6b3c-8130-497b-b850-fca340df911a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5279v\" (UID: \"267a6b3c-8130-497b-b850-fca340df911a\") " pod="openshift-insights/insights-runtime-extractor-5279v" Jul 07 11:12:10.225932 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:10.225801 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tt6v5\" (UniqueName: \"kubernetes.io/projected/267a6b3c-8130-497b-b850-fca340df911a-kube-api-access-tt6v5\") pod \"insights-runtime-extractor-5279v\" (UID: \"267a6b3c-8130-497b-b850-fca340df911a\") " pod="openshift-insights/insights-runtime-extractor-5279v" Jul 07 11:12:10.225932 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:10.225841 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/267a6b3c-8130-497b-b850-fca340df911a-data-volume\") pod \"insights-runtime-extractor-5279v\" (UID: \"267a6b3c-8130-497b-b850-fca340df911a\") " pod="openshift-insights/insights-runtime-extractor-5279v" Jul 07 11:12:10.225932 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:10.225900 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/267a6b3c-8130-497b-b850-fca340df911a-crio-socket\") pod \"insights-runtime-extractor-5279v\" (UID: \"267a6b3c-8130-497b-b850-fca340df911a\") " pod="openshift-insights/insights-runtime-extractor-5279v" Jul 07 11:12:10.226192 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:10.226173 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/267a6b3c-8130-497b-b850-fca340df911a-data-volume\") pod \"insights-runtime-extractor-5279v\" (UID: \"267a6b3c-8130-497b-b850-fca340df911a\") " pod="openshift-insights/insights-runtime-extractor-5279v" Jul 07 11:12:10.226250 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:10.226201 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/267a6b3c-8130-497b-b850-fca340df911a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5279v\" (UID: \"267a6b3c-8130-497b-b850-fca340df911a\") " pod="openshift-insights/insights-runtime-extractor-5279v" Jul 07 11:12:10.227679 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:10.227662 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/267a6b3c-8130-497b-b850-fca340df911a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5279v\" (UID: \"267a6b3c-8130-497b-b850-fca340df911a\") " pod="openshift-insights/insights-runtime-extractor-5279v" Jul 07 11:12:10.234072 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:10.234052 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tt6v5\" (UniqueName: \"kubernetes.io/projected/267a6b3c-8130-497b-b850-fca340df911a-kube-api-access-tt6v5\") pod \"insights-runtime-extractor-5279v\" (UID: \"267a6b3c-8130-497b-b850-fca340df911a\") " pod="openshift-insights/insights-runtime-extractor-5279v" Jul 07 11:12:10.280601 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:10.280575 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:12:10.286484 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:10.286464 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-k8wv9" Jul 07 11:12:10.318603 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:10.318586 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-mmtwk" Jul 07 11:12:10.379840 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:10.379814 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-5279v" Jul 07 11:12:10.488113 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:10.487944 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-k8wv9"] Jul 07 11:12:10.491844 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:10.491818 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-mmtwk"] Jul 07 11:12:10.541550 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:10.541512 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-5279v"] Jul 07 11:12:10.544412 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:12:10.544390 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod267a6b3c_8130_497b_b850_fca340df911a.slice/crio-3b2f741c09d2ff44ebee2f60bf49c36e5e5ff26f6e859840f302902edb3c88e0 WatchSource:0}: Error finding container 3b2f741c09d2ff44ebee2f60bf49c36e5e5ff26f6e859840f302902edb3c88e0: Status 404 returned error can't find the container with id 3b2f741c09d2ff44ebee2f60bf49c36e5e5ff26f6e859840f302902edb3c88e0 Jul 07 11:12:10.761647 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:10.761567 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7ml6d" Jul 07 11:12:10.761647 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:10.761602 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2q46z" Jul 07 11:12:10.761804 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:10.761736 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sc5bl" Jul 07 11:12:10.765210 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:10.765186 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 07 11:12:10.765340 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:10.765240 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-7b27w\"" Jul 07 11:12:10.765340 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:10.765266 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 07 11:12:10.765340 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:10.765241 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 07 11:12:10.765340 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:10.765309 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 07 11:12:10.765585 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:10.765569 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-cn2cx\"" Jul 07 11:12:11.090397 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:11.090311 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5279v" event={"ID":"267a6b3c-8130-497b-b850-fca340df911a","Type":"ContainerStarted","Data":"3b2f741c09d2ff44ebee2f60bf49c36e5e5ff26f6e859840f302902edb3c88e0"} Jul 07 11:12:11.091583 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:11.091553 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-mmtwk" event={"ID":"0bcfaddb-0c6d-4519-a1a7-c23b3a610272","Type":"ContainerStarted","Data":"0b3296fca00dc8d735f73fca4aacbf68268df048255f9cdc2db4480ce2003ede"} Jul 07 11:12:11.092765 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:11.092706 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-k8wv9" event={"ID":"c5da13ef-838f-4deb-9b49-7b12cab2bb84","Type":"ContainerStarted","Data":"48eb3f851f1ae7eb21263f5e859f82ba1d23bbb463c06291c4c35ba62bf54627"} Jul 07 11:12:11.362961 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:11.362854 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:12:11.405605 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:11.404459 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-rpmtl"] Jul 07 11:12:11.417814 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:11.416634 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-rpmtl" Jul 07 11:12:11.417814 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:11.417562 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-rpmtl"] Jul 07 11:12:11.422374 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:11.422193 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 07 11:12:11.423264 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:11.423242 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 07 11:12:11.423486 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:11.423453 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 07 11:12:11.423727 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:11.423709 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 07 11:12:11.424322 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:11.424272 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-mxgbb\"" Jul 07 11:12:11.424520 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:11.424475 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 07 11:12:11.448336 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:11.446674 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:12:11.534744 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:11.534710 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/5bdca415-9aba-484a-9d71-fe763b0e6215-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-rpmtl\" (UID: \"5bdca415-9aba-484a-9d71-fe763b0e6215\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-rpmtl" Jul 07 11:12:11.534904 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:11.534775 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6bxsn\" (UniqueName: \"kubernetes.io/projected/5bdca415-9aba-484a-9d71-fe763b0e6215-kube-api-access-6bxsn\") pod \"prometheus-operator-567f7b8b7c-rpmtl\" (UID: \"5bdca415-9aba-484a-9d71-fe763b0e6215\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-rpmtl" Jul 07 11:12:11.534904 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:11.534811 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5bdca415-9aba-484a-9d71-fe763b0e6215-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-rpmtl\" (UID: \"5bdca415-9aba-484a-9d71-fe763b0e6215\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-rpmtl" Jul 07 11:12:11.534904 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:11.534863 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5bdca415-9aba-484a-9d71-fe763b0e6215-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-rpmtl\" (UID: \"5bdca415-9aba-484a-9d71-fe763b0e6215\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-rpmtl" Jul 07 11:12:11.556848 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:11.556813 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:12:11.635723 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:11.635683 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/5bdca415-9aba-484a-9d71-fe763b0e6215-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-rpmtl\" (UID: \"5bdca415-9aba-484a-9d71-fe763b0e6215\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-rpmtl" Jul 07 11:12:11.635880 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:11.635741 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6bxsn\" (UniqueName: \"kubernetes.io/projected/5bdca415-9aba-484a-9d71-fe763b0e6215-kube-api-access-6bxsn\") pod \"prometheus-operator-567f7b8b7c-rpmtl\" (UID: \"5bdca415-9aba-484a-9d71-fe763b0e6215\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-rpmtl" Jul 07 11:12:11.635880 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:11.635779 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5bdca415-9aba-484a-9d71-fe763b0e6215-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-rpmtl\" (UID: \"5bdca415-9aba-484a-9d71-fe763b0e6215\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-rpmtl" Jul 07 11:12:11.635880 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:11.635806 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5bdca415-9aba-484a-9d71-fe763b0e6215-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-rpmtl\" (UID: \"5bdca415-9aba-484a-9d71-fe763b0e6215\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-rpmtl" Jul 07 11:12:11.637297 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:11.636840 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5bdca415-9aba-484a-9d71-fe763b0e6215-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-rpmtl\" (UID: \"5bdca415-9aba-484a-9d71-fe763b0e6215\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-rpmtl" Jul 07 11:12:11.651955 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:11.650571 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5bdca415-9aba-484a-9d71-fe763b0e6215-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-rpmtl\" (UID: \"5bdca415-9aba-484a-9d71-fe763b0e6215\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-rpmtl" Jul 07 11:12:11.653894 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:11.653865 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6bxsn\" (UniqueName: \"kubernetes.io/projected/5bdca415-9aba-484a-9d71-fe763b0e6215-kube-api-access-6bxsn\") pod \"prometheus-operator-567f7b8b7c-rpmtl\" (UID: \"5bdca415-9aba-484a-9d71-fe763b0e6215\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-rpmtl" Jul 07 11:12:11.655474 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:11.655447 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/5bdca415-9aba-484a-9d71-fe763b0e6215-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-rpmtl\" (UID: \"5bdca415-9aba-484a-9d71-fe763b0e6215\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-rpmtl" Jul 07 11:12:11.737250 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:11.737222 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-rpmtl" Jul 07 11:12:11.930868 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:11.930795 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-rpmtl"] Jul 07 11:12:12.563485 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:12.563449 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:12:12.591713 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:12:12.591677 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5bdca415_9aba_484a_9d71_fe763b0e6215.slice/crio-5c4d3bc8014f00a106a9225dd936d2494e5a5915b2b03f5b811c963a9e21b1bd WatchSource:0}: Error finding container 5c4d3bc8014f00a106a9225dd936d2494e5a5915b2b03f5b811c963a9e21b1bd: Status 404 returned error can't find the container with id 5c4d3bc8014f00a106a9225dd936d2494e5a5915b2b03f5b811c963a9e21b1bd Jul 07 11:12:12.664606 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:12.664577 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:12:13.042101 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:13.041851 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:12:13.101051 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:13.101014 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-k8wv9" event={"ID":"c5da13ef-838f-4deb-9b49-7b12cab2bb84","Type":"ContainerStarted","Data":"d755e87b158fe2fb48cf468eb25d472430dc9b1f90253baf304930caba0d9c08"} Jul 07 11:12:13.102283 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:13.102245 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-rpmtl" event={"ID":"5bdca415-9aba-484a-9d71-fe763b0e6215","Type":"ContainerStarted","Data":"5c4d3bc8014f00a106a9225dd936d2494e5a5915b2b03f5b811c963a9e21b1bd"} Jul 07 11:12:16.111679 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:16.111640 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5279v" event={"ID":"267a6b3c-8130-497b-b850-fca340df911a","Type":"ContainerStarted","Data":"69c962d1591731bdf09a75d41f05b5fc784bab6d6b47c6db2712fe108cc077c9"} Jul 07 11:12:16.113162 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:16.113129 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-mmtwk" event={"ID":"0bcfaddb-0c6d-4519-a1a7-c23b3a610272","Type":"ContainerStarted","Data":"175e13de5e92ca451de2d4faf81ea9cbaccf2765070cbcc55e2f58a3c3f22292"} Jul 07 11:12:16.114722 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:16.114701 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-k8wv9" event={"ID":"c5da13ef-838f-4deb-9b49-7b12cab2bb84","Type":"ContainerStarted","Data":"3638f94a658594eace87d8ec29e88db45edf22011f18ab6df220000c84999214"} Jul 07 11:12:16.115082 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:16.115066 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-k8wv9" Jul 07 11:12:16.116217 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:16.116198 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-rpmtl" event={"ID":"5bdca415-9aba-484a-9d71-fe763b0e6215","Type":"ContainerStarted","Data":"171c393a84704a5f606182221560a3dcf5ddc5fa41ec6510337c2ea33533333e"} Jul 07 11:12:16.146981 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:16.146786 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-k8wv9" podStartSLOduration=5.019625483 podStartE2EDuration="7.146765019s" totalImagesPullingTime="2.127139536s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:12:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:12:16.146541119 +0000 UTC m=+60.560410297" watchObservedRunningTime="2026-07-07 11:12:16.146765019 +0000 UTC m=+60.560634186" Jul 07 11:12:16.147459 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:16.147423 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-mmtwk" podStartSLOduration=1.66971169 podStartE2EDuration="7.147411558s" totalImagesPullingTime="5.477699868s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:12:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:12:16.129968992 +0000 UTC m=+60.543838156" watchObservedRunningTime="2026-07-07 11:12:16.147411558 +0000 UTC m=+60.561280730" Jul 07 11:12:16.300790 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:16.300703 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:12:17.033645 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:17.033597 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-tzf62" Jul 07 11:12:17.123707 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:17.123677 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-rpmtl" event={"ID":"5bdca415-9aba-484a-9d71-fe763b0e6215","Type":"ContainerStarted","Data":"46702193724b5c73d91cb317fc5bde2312de8d2d6cee9efdd4c20e4ed5f838df"} Jul 07 11:12:17.125375 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:17.125353 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5279v" event={"ID":"267a6b3c-8130-497b-b850-fca340df911a","Type":"ContainerStarted","Data":"ece4920f46937c1b1e13515bd4a97a1feed974c1edf4a79695d85780c489aad3"} Jul 07 11:12:17.144166 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:17.144120 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-rpmtl" podStartSLOduration=2.7322884480000003 podStartE2EDuration="6.144108828s" totalImagesPullingTime="3.41182038s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:12:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:12:17.143599613 +0000 UTC m=+61.557468778" watchObservedRunningTime="2026-07-07 11:12:17.144108828 +0000 UTC m=+61.557977992" Jul 07 11:12:17.185273 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:17.185243 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:12:17.261645 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:17.261616 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:12:18.522008 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:18.521941 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:12:18.719558 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:18.719520 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:12:18.749823 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:18.749793 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-kqpvf"] Jul 07 11:12:18.756855 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:18.756832 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-kqpvf" Jul 07 11:12:18.759288 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:18.759237 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 07 11:12:18.759516 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:18.759493 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-rkptn\"" Jul 07 11:12:18.759721 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:18.759704 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 07 11:12:18.765605 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:18.765582 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-kqpvf"] Jul 07 11:12:18.768077 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:18.768054 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-lvbqx"] Jul 07 11:12:18.771355 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:18.771336 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-lvbqx" Jul 07 11:12:18.771722 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:18.771514 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-2ffb8"] Jul 07 11:12:18.773770 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:18.773720 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-2ffb8" Jul 07 11:12:18.773868 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:18.773811 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 07 11:12:18.774608 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:18.774519 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-n8dbl\"" Jul 07 11:12:18.774608 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:18.774591 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 07 11:12:18.774776 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:18.774591 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 07 11:12:18.776452 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:18.776433 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-666wg\"" Jul 07 11:12:18.776668 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:18.776624 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 07 11:12:18.776889 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:18.776860 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 07 11:12:18.777191 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:18.777159 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 07 11:12:18.786350 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:18.786328 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-lvbqx"] Jul 07 11:12:18.790003 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:18.789975 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/2ac7717f-0d38-4ac8-b705-d6b9bcbf7859-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-kqpvf\" (UID: \"2ac7717f-0d38-4ac8-b705-d6b9bcbf7859\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-kqpvf" Jul 07 11:12:18.790107 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:18.790034 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2ac7717f-0d38-4ac8-b705-d6b9bcbf7859-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-kqpvf\" (UID: \"2ac7717f-0d38-4ac8-b705-d6b9bcbf7859\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-kqpvf" Jul 07 11:12:18.790107 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:18.790065 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wgx4n\" (UniqueName: \"kubernetes.io/projected/2ac7717f-0d38-4ac8-b705-d6b9bcbf7859-kube-api-access-wgx4n\") pod \"openshift-state-metrics-54cf96dfcf-kqpvf\" (UID: \"2ac7717f-0d38-4ac8-b705-d6b9bcbf7859\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-kqpvf" Jul 07 11:12:18.790219 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:18.790107 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2ac7717f-0d38-4ac8-b705-d6b9bcbf7859-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-kqpvf\" (UID: \"2ac7717f-0d38-4ac8-b705-d6b9bcbf7859\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-kqpvf" Jul 07 11:12:18.815136 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:18.815108 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:12:18.891062 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:18.891033 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/acd61036-c152-4340-9a8a-cb48eff0f398-metrics-client-ca\") pod \"node-exporter-2ffb8\" (UID: \"acd61036-c152-4340-9a8a-cb48eff0f398\") " pod="openshift-monitoring/node-exporter-2ffb8" Jul 07 11:12:18.891209 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:18.891082 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/124f8526-2fa2-4206-817b-85d6b0d00d56-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-lvbqx\" (UID: \"124f8526-2fa2-4206-817b-85d6b0d00d56\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-lvbqx" Jul 07 11:12:18.891209 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:18.891114 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rj7wt\" (UniqueName: \"kubernetes.io/projected/124f8526-2fa2-4206-817b-85d6b0d00d56-kube-api-access-rj7wt\") pod \"kube-state-metrics-7c9dbb8b7d-lvbqx\" (UID: \"124f8526-2fa2-4206-817b-85d6b0d00d56\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-lvbqx" Jul 07 11:12:18.891209 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:18.891143 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h8lrm\" (UniqueName: \"kubernetes.io/projected/acd61036-c152-4340-9a8a-cb48eff0f398-kube-api-access-h8lrm\") pod \"node-exporter-2ffb8\" (UID: \"acd61036-c152-4340-9a8a-cb48eff0f398\") " pod="openshift-monitoring/node-exporter-2ffb8" Jul 07 11:12:18.891209 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:18.891200 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/acd61036-c152-4340-9a8a-cb48eff0f398-sys\") pod \"node-exporter-2ffb8\" (UID: \"acd61036-c152-4340-9a8a-cb48eff0f398\") " pod="openshift-monitoring/node-exporter-2ffb8" Jul 07 11:12:18.891384 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:18.891232 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2ac7717f-0d38-4ac8-b705-d6b9bcbf7859-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-kqpvf\" (UID: \"2ac7717f-0d38-4ac8-b705-d6b9bcbf7859\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-kqpvf" Jul 07 11:12:18.891384 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:18.891253 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wgx4n\" (UniqueName: \"kubernetes.io/projected/2ac7717f-0d38-4ac8-b705-d6b9bcbf7859-kube-api-access-wgx4n\") pod \"openshift-state-metrics-54cf96dfcf-kqpvf\" (UID: \"2ac7717f-0d38-4ac8-b705-d6b9bcbf7859\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-kqpvf" Jul 07 11:12:18.891384 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:18.891303 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/124f8526-2fa2-4206-817b-85d6b0d00d56-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-lvbqx\" (UID: \"124f8526-2fa2-4206-817b-85d6b0d00d56\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-lvbqx" Jul 07 11:12:18.891384 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:18.891336 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/acd61036-c152-4340-9a8a-cb48eff0f398-node-exporter-textfile\") pod \"node-exporter-2ffb8\" (UID: \"acd61036-c152-4340-9a8a-cb48eff0f398\") " pod="openshift-monitoring/node-exporter-2ffb8" Jul 07 11:12:18.891384 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:18.891361 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/124f8526-2fa2-4206-817b-85d6b0d00d56-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-lvbqx\" (UID: \"124f8526-2fa2-4206-817b-85d6b0d00d56\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-lvbqx" Jul 07 11:12:18.891548 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:18.891392 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/acd61036-c152-4340-9a8a-cb48eff0f398-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-2ffb8\" (UID: \"acd61036-c152-4340-9a8a-cb48eff0f398\") " pod="openshift-monitoring/node-exporter-2ffb8" Jul 07 11:12:18.891548 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:18.891415 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/124f8526-2fa2-4206-817b-85d6b0d00d56-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-lvbqx\" (UID: \"124f8526-2fa2-4206-817b-85d6b0d00d56\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-lvbqx" Jul 07 11:12:18.891548 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:18.891453 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/acd61036-c152-4340-9a8a-cb48eff0f398-root\") pod \"node-exporter-2ffb8\" (UID: \"acd61036-c152-4340-9a8a-cb48eff0f398\") " pod="openshift-monitoring/node-exporter-2ffb8" Jul 07 11:12:18.891548 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:18.891492 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/acd61036-c152-4340-9a8a-cb48eff0f398-node-exporter-accelerators-collector-config\") pod \"node-exporter-2ffb8\" (UID: \"acd61036-c152-4340-9a8a-cb48eff0f398\") " pod="openshift-monitoring/node-exporter-2ffb8" Jul 07 11:12:18.891750 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:18.891648 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/acd61036-c152-4340-9a8a-cb48eff0f398-node-exporter-tls\") pod \"node-exporter-2ffb8\" (UID: \"acd61036-c152-4340-9a8a-cb48eff0f398\") " pod="openshift-monitoring/node-exporter-2ffb8" Jul 07 11:12:18.891750 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:18.891700 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/2ac7717f-0d38-4ac8-b705-d6b9bcbf7859-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-kqpvf\" (UID: \"2ac7717f-0d38-4ac8-b705-d6b9bcbf7859\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-kqpvf" Jul 07 11:12:18.891750 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:18.891728 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/acd61036-c152-4340-9a8a-cb48eff0f398-node-exporter-wtmp\") pod \"node-exporter-2ffb8\" (UID: \"acd61036-c152-4340-9a8a-cb48eff0f398\") " pod="openshift-monitoring/node-exporter-2ffb8" Jul 07 11:12:18.891874 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:18.891755 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/124f8526-2fa2-4206-817b-85d6b0d00d56-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-lvbqx\" (UID: \"124f8526-2fa2-4206-817b-85d6b0d00d56\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-lvbqx" Jul 07 11:12:18.891874 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:18.891794 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2ac7717f-0d38-4ac8-b705-d6b9bcbf7859-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-kqpvf\" (UID: \"2ac7717f-0d38-4ac8-b705-d6b9bcbf7859\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-kqpvf" Jul 07 11:12:18.891874 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:12:18.891829 2562 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found Jul 07 11:12:18.892059 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:12:18.891909 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ac7717f-0d38-4ac8-b705-d6b9bcbf7859-openshift-state-metrics-tls podName:2ac7717f-0d38-4ac8-b705-d6b9bcbf7859 nodeName:}" failed. No retries permitted until 2026-07-07 11:12:19.39188968 +0000 UTC m=+63.805758836 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/2ac7717f-0d38-4ac8-b705-d6b9bcbf7859-openshift-state-metrics-tls") pod "openshift-state-metrics-54cf96dfcf-kqpvf" (UID: "2ac7717f-0d38-4ac8-b705-d6b9bcbf7859") : secret "openshift-state-metrics-tls" not found Jul 07 11:12:18.892584 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:18.892556 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2ac7717f-0d38-4ac8-b705-d6b9bcbf7859-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-kqpvf\" (UID: \"2ac7717f-0d38-4ac8-b705-d6b9bcbf7859\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-kqpvf" Jul 07 11:12:18.894467 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:18.894444 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2ac7717f-0d38-4ac8-b705-d6b9bcbf7859-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-kqpvf\" (UID: \"2ac7717f-0d38-4ac8-b705-d6b9bcbf7859\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-kqpvf" Jul 07 11:12:18.897794 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:18.897768 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:12:18.907371 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:18.907349 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wgx4n\" (UniqueName: \"kubernetes.io/projected/2ac7717f-0d38-4ac8-b705-d6b9bcbf7859-kube-api-access-wgx4n\") pod \"openshift-state-metrics-54cf96dfcf-kqpvf\" (UID: \"2ac7717f-0d38-4ac8-b705-d6b9bcbf7859\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-kqpvf" Jul 07 11:12:18.992839 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:18.992807 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/124f8526-2fa2-4206-817b-85d6b0d00d56-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-lvbqx\" (UID: \"124f8526-2fa2-4206-817b-85d6b0d00d56\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-lvbqx" Jul 07 11:12:18.993233 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:18.992862 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/acd61036-c152-4340-9a8a-cb48eff0f398-node-exporter-textfile\") pod \"node-exporter-2ffb8\" (UID: \"acd61036-c152-4340-9a8a-cb48eff0f398\") " pod="openshift-monitoring/node-exporter-2ffb8" Jul 07 11:12:18.993233 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:18.992893 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/124f8526-2fa2-4206-817b-85d6b0d00d56-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-lvbqx\" (UID: \"124f8526-2fa2-4206-817b-85d6b0d00d56\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-lvbqx" Jul 07 11:12:18.993233 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:18.993010 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/acd61036-c152-4340-9a8a-cb48eff0f398-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-2ffb8\" (UID: \"acd61036-c152-4340-9a8a-cb48eff0f398\") " pod="openshift-monitoring/node-exporter-2ffb8" Jul 07 11:12:18.993233 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:18.993071 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/124f8526-2fa2-4206-817b-85d6b0d00d56-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-lvbqx\" (UID: \"124f8526-2fa2-4206-817b-85d6b0d00d56\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-lvbqx" Jul 07 11:12:18.993233 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:18.993110 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/acd61036-c152-4340-9a8a-cb48eff0f398-root\") pod \"node-exporter-2ffb8\" (UID: \"acd61036-c152-4340-9a8a-cb48eff0f398\") " pod="openshift-monitoring/node-exporter-2ffb8" Jul 07 11:12:18.993233 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:18.993153 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/acd61036-c152-4340-9a8a-cb48eff0f398-node-exporter-accelerators-collector-config\") pod \"node-exporter-2ffb8\" (UID: \"acd61036-c152-4340-9a8a-cb48eff0f398\") " pod="openshift-monitoring/node-exporter-2ffb8" Jul 07 11:12:18.993233 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:18.993209 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/acd61036-c152-4340-9a8a-cb48eff0f398-node-exporter-tls\") pod \"node-exporter-2ffb8\" (UID: \"acd61036-c152-4340-9a8a-cb48eff0f398\") " pod="openshift-monitoring/node-exporter-2ffb8" Jul 07 11:12:18.993993 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:18.993250 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/acd61036-c152-4340-9a8a-cb48eff0f398-node-exporter-wtmp\") pod \"node-exporter-2ffb8\" (UID: \"acd61036-c152-4340-9a8a-cb48eff0f398\") " pod="openshift-monitoring/node-exporter-2ffb8" Jul 07 11:12:18.993993 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:18.993273 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/124f8526-2fa2-4206-817b-85d6b0d00d56-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-lvbqx\" (UID: \"124f8526-2fa2-4206-817b-85d6b0d00d56\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-lvbqx" Jul 07 11:12:18.993993 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:18.993316 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/acd61036-c152-4340-9a8a-cb48eff0f398-metrics-client-ca\") pod \"node-exporter-2ffb8\" (UID: \"acd61036-c152-4340-9a8a-cb48eff0f398\") " pod="openshift-monitoring/node-exporter-2ffb8" Jul 07 11:12:18.993993 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:18.993349 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/124f8526-2fa2-4206-817b-85d6b0d00d56-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-lvbqx\" (UID: \"124f8526-2fa2-4206-817b-85d6b0d00d56\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-lvbqx" Jul 07 11:12:18.993993 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:18.993375 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rj7wt\" (UniqueName: \"kubernetes.io/projected/124f8526-2fa2-4206-817b-85d6b0d00d56-kube-api-access-rj7wt\") pod \"kube-state-metrics-7c9dbb8b7d-lvbqx\" (UID: \"124f8526-2fa2-4206-817b-85d6b0d00d56\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-lvbqx" Jul 07 11:12:18.993993 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:18.993406 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h8lrm\" (UniqueName: \"kubernetes.io/projected/acd61036-c152-4340-9a8a-cb48eff0f398-kube-api-access-h8lrm\") pod \"node-exporter-2ffb8\" (UID: \"acd61036-c152-4340-9a8a-cb48eff0f398\") " pod="openshift-monitoring/node-exporter-2ffb8" Jul 07 11:12:18.993993 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:18.993469 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/acd61036-c152-4340-9a8a-cb48eff0f398-sys\") pod \"node-exporter-2ffb8\" (UID: \"acd61036-c152-4340-9a8a-cb48eff0f398\") " pod="openshift-monitoring/node-exporter-2ffb8" Jul 07 11:12:18.993993 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:18.993623 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/124f8526-2fa2-4206-817b-85d6b0d00d56-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-lvbqx\" (UID: \"124f8526-2fa2-4206-817b-85d6b0d00d56\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-lvbqx" Jul 07 11:12:18.993993 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:18.993634 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/acd61036-c152-4340-9a8a-cb48eff0f398-sys\") pod \"node-exporter-2ffb8\" (UID: \"acd61036-c152-4340-9a8a-cb48eff0f398\") " pod="openshift-monitoring/node-exporter-2ffb8" Jul 07 11:12:18.993993 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:18.993694 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/acd61036-c152-4340-9a8a-cb48eff0f398-node-exporter-textfile\") pod \"node-exporter-2ffb8\" (UID: \"acd61036-c152-4340-9a8a-cb48eff0f398\") " pod="openshift-monitoring/node-exporter-2ffb8" Jul 07 11:12:18.993993 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:18.993718 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/124f8526-2fa2-4206-817b-85d6b0d00d56-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-lvbqx\" (UID: \"124f8526-2fa2-4206-817b-85d6b0d00d56\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-lvbqx" Jul 07 11:12:18.993993 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:18.993745 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/acd61036-c152-4340-9a8a-cb48eff0f398-node-exporter-wtmp\") pod \"node-exporter-2ffb8\" (UID: \"acd61036-c152-4340-9a8a-cb48eff0f398\") " pod="openshift-monitoring/node-exporter-2ffb8" Jul 07 11:12:18.993993 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:18.993980 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/124f8526-2fa2-4206-817b-85d6b0d00d56-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-lvbqx\" (UID: \"124f8526-2fa2-4206-817b-85d6b0d00d56\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-lvbqx" Jul 07 11:12:18.994673 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:18.994036 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/acd61036-c152-4340-9a8a-cb48eff0f398-root\") pod \"node-exporter-2ffb8\" (UID: \"acd61036-c152-4340-9a8a-cb48eff0f398\") " pod="openshift-monitoring/node-exporter-2ffb8" Jul 07 11:12:18.994673 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:18.994485 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/acd61036-c152-4340-9a8a-cb48eff0f398-node-exporter-accelerators-collector-config\") pod \"node-exporter-2ffb8\" (UID: \"acd61036-c152-4340-9a8a-cb48eff0f398\") " pod="openshift-monitoring/node-exporter-2ffb8" Jul 07 11:12:18.994673 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:18.994532 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/acd61036-c152-4340-9a8a-cb48eff0f398-metrics-client-ca\") pod \"node-exporter-2ffb8\" (UID: \"acd61036-c152-4340-9a8a-cb48eff0f398\") " pod="openshift-monitoring/node-exporter-2ffb8" Jul 07 11:12:18.996827 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:18.996799 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:12:18.997981 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:18.997866 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/acd61036-c152-4340-9a8a-cb48eff0f398-node-exporter-tls\") pod \"node-exporter-2ffb8\" (UID: \"acd61036-c152-4340-9a8a-cb48eff0f398\") " pod="openshift-monitoring/node-exporter-2ffb8" Jul 07 11:12:18.998081 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:18.998005 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/acd61036-c152-4340-9a8a-cb48eff0f398-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-2ffb8\" (UID: \"acd61036-c152-4340-9a8a-cb48eff0f398\") " pod="openshift-monitoring/node-exporter-2ffb8" Jul 07 11:12:18.998224 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:18.998194 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/124f8526-2fa2-4206-817b-85d6b0d00d56-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-lvbqx\" (UID: \"124f8526-2fa2-4206-817b-85d6b0d00d56\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-lvbqx" Jul 07 11:12:18.999597 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:18.999574 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/124f8526-2fa2-4206-817b-85d6b0d00d56-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-lvbqx\" (UID: \"124f8526-2fa2-4206-817b-85d6b0d00d56\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-lvbqx" Jul 07 11:12:19.002808 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:19.002786 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h8lrm\" (UniqueName: \"kubernetes.io/projected/acd61036-c152-4340-9a8a-cb48eff0f398-kube-api-access-h8lrm\") pod \"node-exporter-2ffb8\" (UID: \"acd61036-c152-4340-9a8a-cb48eff0f398\") " pod="openshift-monitoring/node-exporter-2ffb8" Jul 07 11:12:19.003342 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:19.003323 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rj7wt\" (UniqueName: \"kubernetes.io/projected/124f8526-2fa2-4206-817b-85d6b0d00d56-kube-api-access-rj7wt\") pod \"kube-state-metrics-7c9dbb8b7d-lvbqx\" (UID: \"124f8526-2fa2-4206-817b-85d6b0d00d56\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-lvbqx" Jul 07 11:12:19.084099 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:19.084036 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-lvbqx" Jul 07 11:12:19.090188 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:19.090162 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-2ffb8" Jul 07 11:12:19.094462 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:19.094436 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:12:19.195751 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:12:19.195729 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podacd61036_c152_4340_9a8a_cb48eff0f398.slice/crio-dec855769d46145dfcd76c39f9398d2efb97a1e37ec5223d17f81ed38b56f39f WatchSource:0}: Error finding container dec855769d46145dfcd76c39f9398d2efb97a1e37ec5223d17f81ed38b56f39f: Status 404 returned error can't find the container with id dec855769d46145dfcd76c39f9398d2efb97a1e37ec5223d17f81ed38b56f39f Jul 07 11:12:19.199510 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:19.199487 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:12:19.295809 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:19.295787 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:12:19.310934 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:19.310898 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-lvbqx"] Jul 07 11:12:19.314114 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:12:19.314093 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod124f8526_2fa2_4206_817b_85d6b0d00d56.slice/crio-e7057d3266dbb52dbf738c60dfd8859ab6a7c61fbea9deb4719d77aecec6d671 WatchSource:0}: Error finding container e7057d3266dbb52dbf738c60dfd8859ab6a7c61fbea9deb4719d77aecec6d671: Status 404 returned error can't find the container with id e7057d3266dbb52dbf738c60dfd8859ab6a7c61fbea9deb4719d77aecec6d671 Jul 07 11:12:19.395957 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:19.395939 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/2ac7717f-0d38-4ac8-b705-d6b9bcbf7859-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-kqpvf\" (UID: \"2ac7717f-0d38-4ac8-b705-d6b9bcbf7859\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-kqpvf" Jul 07 11:12:19.397908 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:19.397893 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/2ac7717f-0d38-4ac8-b705-d6b9bcbf7859-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-kqpvf\" (UID: \"2ac7717f-0d38-4ac8-b705-d6b9bcbf7859\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-kqpvf" Jul 07 11:12:19.668858 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:19.668792 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-kqpvf" Jul 07 11:12:19.745057 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:19.745025 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:12:19.809202 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:19.809175 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-kqpvf"] Jul 07 11:12:19.812277 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:12:19.812250 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ac7717f_0d38_4ac8_b705_d6b9bcbf7859.slice/crio-126ecf8b404225239908cd2f1bd567ec10b3b569a9586fb9ffd0270a7e056e9c WatchSource:0}: Error finding container 126ecf8b404225239908cd2f1bd567ec10b3b569a9586fb9ffd0270a7e056e9c: Status 404 returned error can't find the container with id 126ecf8b404225239908cd2f1bd567ec10b3b569a9586fb9ffd0270a7e056e9c Jul 07 11:12:19.847339 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:19.847227 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:12:19.852595 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:19.852385 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 07 11:12:19.855673 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:19.855444 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:12:19.858895 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:19.857992 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 07 11:12:19.858895 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:19.858083 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 07 11:12:19.858895 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:19.858204 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-nlp2c\"" Jul 07 11:12:19.858895 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:19.858386 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 07 11:12:19.858895 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:19.858393 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 07 11:12:19.858895 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:19.858544 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 07 11:12:19.858895 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:19.858622 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 07 11:12:19.859908 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:19.859469 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 07 11:12:19.859908 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:19.859667 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 07 11:12:19.859908 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:19.859846 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 07 11:12:19.871931 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:19.871881 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 07 11:12:19.903318 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:19.903287 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9314229f-a3c5-4809-8129-b06de0b52127-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"9314229f-a3c5-4809-8129-b06de0b52127\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:12:19.903458 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:19.903350 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/9314229f-a3c5-4809-8129-b06de0b52127-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"9314229f-a3c5-4809-8129-b06de0b52127\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:12:19.903458 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:19.903416 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9314229f-a3c5-4809-8129-b06de0b52127-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"9314229f-a3c5-4809-8129-b06de0b52127\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:12:19.903458 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:19.903441 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9314229f-a3c5-4809-8129-b06de0b52127-tls-assets\") pod \"alertmanager-main-0\" (UID: \"9314229f-a3c5-4809-8129-b06de0b52127\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:12:19.903615 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:19.903470 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9314229f-a3c5-4809-8129-b06de0b52127-web-config\") pod \"alertmanager-main-0\" (UID: \"9314229f-a3c5-4809-8129-b06de0b52127\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:12:19.903615 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:19.903496 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/9314229f-a3c5-4809-8129-b06de0b52127-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"9314229f-a3c5-4809-8129-b06de0b52127\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:12:19.903615 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:19.903531 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/9314229f-a3c5-4809-8129-b06de0b52127-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"9314229f-a3c5-4809-8129-b06de0b52127\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:12:19.903615 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:19.903557 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/9314229f-a3c5-4809-8129-b06de0b52127-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"9314229f-a3c5-4809-8129-b06de0b52127\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:12:19.903615 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:19.903583 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dtqff\" (UniqueName: \"kubernetes.io/projected/9314229f-a3c5-4809-8129-b06de0b52127-kube-api-access-dtqff\") pod \"alertmanager-main-0\" (UID: \"9314229f-a3c5-4809-8129-b06de0b52127\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:12:19.903615 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:19.903612 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9314229f-a3c5-4809-8129-b06de0b52127-config-out\") pod \"alertmanager-main-0\" (UID: \"9314229f-a3c5-4809-8129-b06de0b52127\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:12:19.903887 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:19.903645 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9314229f-a3c5-4809-8129-b06de0b52127-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"9314229f-a3c5-4809-8129-b06de0b52127\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:12:19.903887 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:19.903680 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/9314229f-a3c5-4809-8129-b06de0b52127-config-volume\") pod \"alertmanager-main-0\" (UID: \"9314229f-a3c5-4809-8129-b06de0b52127\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:12:19.903887 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:19.903707 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9314229f-a3c5-4809-8129-b06de0b52127-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"9314229f-a3c5-4809-8129-b06de0b52127\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:12:20.004544 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:20.004474 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/9314229f-a3c5-4809-8129-b06de0b52127-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"9314229f-a3c5-4809-8129-b06de0b52127\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:12:20.004544 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:20.004534 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9314229f-a3c5-4809-8129-b06de0b52127-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"9314229f-a3c5-4809-8129-b06de0b52127\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:12:20.004753 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:20.004553 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9314229f-a3c5-4809-8129-b06de0b52127-tls-assets\") pod \"alertmanager-main-0\" (UID: \"9314229f-a3c5-4809-8129-b06de0b52127\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:12:20.004753 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:20.004586 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9314229f-a3c5-4809-8129-b06de0b52127-web-config\") pod \"alertmanager-main-0\" (UID: \"9314229f-a3c5-4809-8129-b06de0b52127\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:12:20.004753 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:20.004611 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/9314229f-a3c5-4809-8129-b06de0b52127-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"9314229f-a3c5-4809-8129-b06de0b52127\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:12:20.004899 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:20.004764 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/9314229f-a3c5-4809-8129-b06de0b52127-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"9314229f-a3c5-4809-8129-b06de0b52127\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:12:20.004899 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:20.004804 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/9314229f-a3c5-4809-8129-b06de0b52127-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"9314229f-a3c5-4809-8129-b06de0b52127\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:12:20.004899 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:20.004834 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dtqff\" (UniqueName: \"kubernetes.io/projected/9314229f-a3c5-4809-8129-b06de0b52127-kube-api-access-dtqff\") pod \"alertmanager-main-0\" (UID: \"9314229f-a3c5-4809-8129-b06de0b52127\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:12:20.004899 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:20.004864 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9314229f-a3c5-4809-8129-b06de0b52127-config-out\") pod \"alertmanager-main-0\" (UID: \"9314229f-a3c5-4809-8129-b06de0b52127\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:12:20.005117 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:20.004902 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9314229f-a3c5-4809-8129-b06de0b52127-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"9314229f-a3c5-4809-8129-b06de0b52127\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:12:20.005117 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:20.004959 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/9314229f-a3c5-4809-8129-b06de0b52127-config-volume\") pod \"alertmanager-main-0\" (UID: \"9314229f-a3c5-4809-8129-b06de0b52127\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:12:20.005117 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:20.004974 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/9314229f-a3c5-4809-8129-b06de0b52127-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"9314229f-a3c5-4809-8129-b06de0b52127\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:12:20.005117 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:20.004989 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9314229f-a3c5-4809-8129-b06de0b52127-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"9314229f-a3c5-4809-8129-b06de0b52127\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:12:20.005117 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:20.005060 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9314229f-a3c5-4809-8129-b06de0b52127-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"9314229f-a3c5-4809-8129-b06de0b52127\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:12:20.005479 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:12:20.005365 2562 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found Jul 07 11:12:20.005479 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:12:20.005433 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9314229f-a3c5-4809-8129-b06de0b52127-secret-alertmanager-main-tls podName:9314229f-a3c5-4809-8129-b06de0b52127 nodeName:}" failed. No retries permitted until 2026-07-07 11:12:20.505413665 +0000 UTC m=+64.919282808 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/9314229f-a3c5-4809-8129-b06de0b52127-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "9314229f-a3c5-4809-8129-b06de0b52127") : secret "alertmanager-main-tls" not found Jul 07 11:12:20.005820 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:20.005780 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9314229f-a3c5-4809-8129-b06de0b52127-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"9314229f-a3c5-4809-8129-b06de0b52127\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:12:20.011956 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:20.008811 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9314229f-a3c5-4809-8129-b06de0b52127-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"9314229f-a3c5-4809-8129-b06de0b52127\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:12:20.011956 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:20.009077 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9314229f-a3c5-4809-8129-b06de0b52127-config-out\") pod \"alertmanager-main-0\" (UID: \"9314229f-a3c5-4809-8129-b06de0b52127\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:12:20.011956 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:20.009326 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9314229f-a3c5-4809-8129-b06de0b52127-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"9314229f-a3c5-4809-8129-b06de0b52127\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:12:20.011956 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:20.010070 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/9314229f-a3c5-4809-8129-b06de0b52127-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"9314229f-a3c5-4809-8129-b06de0b52127\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:12:20.014791 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:20.014768 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/9314229f-a3c5-4809-8129-b06de0b52127-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"9314229f-a3c5-4809-8129-b06de0b52127\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:12:20.014903 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:20.014872 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/9314229f-a3c5-4809-8129-b06de0b52127-config-volume\") pod \"alertmanager-main-0\" (UID: \"9314229f-a3c5-4809-8129-b06de0b52127\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:12:20.014981 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:20.014961 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9314229f-a3c5-4809-8129-b06de0b52127-web-config\") pod \"alertmanager-main-0\" (UID: \"9314229f-a3c5-4809-8129-b06de0b52127\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:12:20.017230 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:20.017204 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9314229f-a3c5-4809-8129-b06de0b52127-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"9314229f-a3c5-4809-8129-b06de0b52127\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:12:20.017589 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:20.017472 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9314229f-a3c5-4809-8129-b06de0b52127-tls-assets\") pod \"alertmanager-main-0\" (UID: \"9314229f-a3c5-4809-8129-b06de0b52127\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:12:20.017908 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:20.017871 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dtqff\" (UniqueName: \"kubernetes.io/projected/9314229f-a3c5-4809-8129-b06de0b52127-kube-api-access-dtqff\") pod \"alertmanager-main-0\" (UID: \"9314229f-a3c5-4809-8129-b06de0b52127\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:12:20.140851 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:20.140308 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5279v" event={"ID":"267a6b3c-8130-497b-b850-fca340df911a","Type":"ContainerStarted","Data":"116750ddb9288d31c751fcc04e799049ae266bcbcb320927ccdc3510d117f244"} Jul 07 11:12:20.141879 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:20.141851 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-lvbqx" event={"ID":"124f8526-2fa2-4206-817b-85d6b0d00d56","Type":"ContainerStarted","Data":"e7057d3266dbb52dbf738c60dfd8859ab6a7c61fbea9deb4719d77aecec6d671"} Jul 07 11:12:20.144053 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:20.144026 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-2ffb8" event={"ID":"acd61036-c152-4340-9a8a-cb48eff0f398","Type":"ContainerStarted","Data":"dec855769d46145dfcd76c39f9398d2efb97a1e37ec5223d17f81ed38b56f39f"} Jul 07 11:12:20.146007 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:20.145982 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-kqpvf" event={"ID":"2ac7717f-0d38-4ac8-b705-d6b9bcbf7859","Type":"ContainerStarted","Data":"17e12722329681fa509285cdeee6ab18ab7eccb94dcfa40a20595d66d8a138bb"} Jul 07 11:12:20.146102 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:20.146038 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-kqpvf" event={"ID":"2ac7717f-0d38-4ac8-b705-d6b9bcbf7859","Type":"ContainerStarted","Data":"126ecf8b404225239908cd2f1bd567ec10b3b569a9586fb9ffd0270a7e056e9c"} Jul 07 11:12:20.159927 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:20.159863 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-5279v" podStartSLOduration=1.631266726 podStartE2EDuration="10.159847323s" totalImagesPullingTime="8.528580597s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 11:12:10 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:12:20.159229906 +0000 UTC m=+64.573099065" watchObservedRunningTime="2026-07-07 11:12:20.159847323 +0000 UTC m=+64.573716489" Jul 07 11:12:20.163590 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:20.163528 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:12:20.510420 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:20.510390 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/9314229f-a3c5-4809-8129-b06de0b52127-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"9314229f-a3c5-4809-8129-b06de0b52127\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:12:20.512814 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:20.512792 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/9314229f-a3c5-4809-8129-b06de0b52127-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"9314229f-a3c5-4809-8129-b06de0b52127\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:12:20.767366 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:20.767341 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:12:20.799130 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:20.799044 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:12:20.900010 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:20.899949 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:12:20.944213 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:20.944062 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 07 11:12:20.946568 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:12:20.946540 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9314229f_a3c5_4809_8129_b06de0b52127.slice/crio-8ba017bb8889f7c2be39bf401be69f96d4b9e92a2af815a30151bbb36e1fe4f1 WatchSource:0}: Error finding container 8ba017bb8889f7c2be39bf401be69f96d4b9e92a2af815a30151bbb36e1fe4f1: Status 404 returned error can't find the container with id 8ba017bb8889f7c2be39bf401be69f96d4b9e92a2af815a30151bbb36e1fe4f1 Jul 07 11:12:21.153292 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:21.153251 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-lvbqx" event={"ID":"124f8526-2fa2-4206-817b-85d6b0d00d56","Type":"ContainerStarted","Data":"d2719bce8f32624be4de35ee37cb63477f61ec8194d619b79c524d06c70f3c3c"} Jul 07 11:12:21.153468 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:21.153323 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-lvbqx" event={"ID":"124f8526-2fa2-4206-817b-85d6b0d00d56","Type":"ContainerStarted","Data":"e41b0e797f93eb0b4a5a7fd3e35ef9f44cf1406bcdec234635928a8c267a59b7"} Jul 07 11:12:21.153468 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:21.153354 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-lvbqx" event={"ID":"124f8526-2fa2-4206-817b-85d6b0d00d56","Type":"ContainerStarted","Data":"5495a0066b7a627b372d53bc14b280b4ffc06224ba3c4a2803a9a5d01b850c0b"} Jul 07 11:12:21.155019 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:21.154991 2562 generic.go:356] "Generic (PLEG): container finished" podID="acd61036-c152-4340-9a8a-cb48eff0f398" containerID="c1d915757b9719837528cde8cd05f33df0907c36b6e7cb1c3ea24a9d2118e61a" exitCode=0 Jul 07 11:12:21.155154 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:21.155071 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-2ffb8" event={"ID":"acd61036-c152-4340-9a8a-cb48eff0f398","Type":"ContainerDied","Data":"c1d915757b9719837528cde8cd05f33df0907c36b6e7cb1c3ea24a9d2118e61a"} Jul 07 11:12:21.156499 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:21.156469 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9314229f-a3c5-4809-8129-b06de0b52127","Type":"ContainerStarted","Data":"8ba017bb8889f7c2be39bf401be69f96d4b9e92a2af815a30151bbb36e1fe4f1"} Jul 07 11:12:21.158683 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:21.158665 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-kqpvf" event={"ID":"2ac7717f-0d38-4ac8-b705-d6b9bcbf7859","Type":"ContainerStarted","Data":"6587720cebc9c913ed1202b3b6fb38944ea6e261001e6d6b873b7a72cedb73ba"} Jul 07 11:12:21.174492 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:21.174453 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-lvbqx" podStartSLOduration=1.883674165 podStartE2EDuration="3.174441036s" totalImagesPullingTime="1.290766871s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:12:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:12:21.173356897 +0000 UTC m=+65.587226063" watchObservedRunningTime="2026-07-07 11:12:21.174441036 +0000 UTC m=+65.588310201" Jul 07 11:12:21.770653 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:21.770562 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:12:21.821956 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:21.821903 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/94c776b2-5ce9-4210-adf5-31efabcef2af-metrics-certs\") pod \"network-metrics-daemon-2q46z\" (UID: \"94c776b2-5ce9-4210-adf5-31efabcef2af\") " pod="openshift-multus/network-metrics-daemon-2q46z" Jul 07 11:12:21.824452 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:21.824426 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 07 11:12:21.834829 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:21.834809 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/94c776b2-5ce9-4210-adf5-31efabcef2af-metrics-certs\") pod \"network-metrics-daemon-2q46z\" (UID: \"94c776b2-5ce9-4210-adf5-31efabcef2af\") " pod="openshift-multus/network-metrics-daemon-2q46z" Jul 07 11:12:21.855272 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:21.855254 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-746f69848d-65tjq"] Jul 07 11:12:21.857513 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:21.857498 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-746f69848d-65tjq" Jul 07 11:12:21.860158 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:21.860128 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 07 11:12:21.860263 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:21.860177 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 07 11:12:21.860263 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:21.860232 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:12:21.860378 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:21.860283 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 07 11:12:21.860378 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:21.860305 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-e7dhd441umo23\"" Jul 07 11:12:21.860378 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:21.860364 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-cp8wv\"" Jul 07 11:12:21.860563 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:21.860548 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 07 11:12:21.860662 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:21.860646 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 07 11:12:21.879384 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:21.879364 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-746f69848d-65tjq"] Jul 07 11:12:21.881961 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:21.881933 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-7b27w\"" Jul 07 11:12:21.888404 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:21.888387 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2q46z" Jul 07 11:12:21.923565 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:21.923536 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/3e27d653-3f08-4102-b75c-ac10382e1959-secret-thanos-querier-tls\") pod \"thanos-querier-746f69848d-65tjq\" (UID: \"3e27d653-3f08-4102-b75c-ac10382e1959\") " pod="openshift-monitoring/thanos-querier-746f69848d-65tjq" Jul 07 11:12:21.923683 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:21.923592 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/3e27d653-3f08-4102-b75c-ac10382e1959-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-746f69848d-65tjq\" (UID: \"3e27d653-3f08-4102-b75c-ac10382e1959\") " pod="openshift-monitoring/thanos-querier-746f69848d-65tjq" Jul 07 11:12:21.923752 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:21.923681 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pk65d\" (UniqueName: \"kubernetes.io/projected/3e27d653-3f08-4102-b75c-ac10382e1959-kube-api-access-pk65d\") pod \"thanos-querier-746f69848d-65tjq\" (UID: \"3e27d653-3f08-4102-b75c-ac10382e1959\") " pod="openshift-monitoring/thanos-querier-746f69848d-65tjq" Jul 07 11:12:21.923752 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:21.923740 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3e27d653-3f08-4102-b75c-ac10382e1959-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-746f69848d-65tjq\" (UID: \"3e27d653-3f08-4102-b75c-ac10382e1959\") " pod="openshift-monitoring/thanos-querier-746f69848d-65tjq" Jul 07 11:12:21.923856 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:21.923799 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3e27d653-3f08-4102-b75c-ac10382e1959-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-746f69848d-65tjq\" (UID: \"3e27d653-3f08-4102-b75c-ac10382e1959\") " pod="openshift-monitoring/thanos-querier-746f69848d-65tjq" Jul 07 11:12:21.923856 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:21.923826 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/3e27d653-3f08-4102-b75c-ac10382e1959-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-746f69848d-65tjq\" (UID: \"3e27d653-3f08-4102-b75c-ac10382e1959\") " pod="openshift-monitoring/thanos-querier-746f69848d-65tjq" Jul 07 11:12:21.923989 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:21.923879 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3e27d653-3f08-4102-b75c-ac10382e1959-secret-grpc-tls\") pod \"thanos-querier-746f69848d-65tjq\" (UID: \"3e27d653-3f08-4102-b75c-ac10382e1959\") " pod="openshift-monitoring/thanos-querier-746f69848d-65tjq" Jul 07 11:12:21.923989 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:21.923947 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3e27d653-3f08-4102-b75c-ac10382e1959-metrics-client-ca\") pod \"thanos-querier-746f69848d-65tjq\" (UID: \"3e27d653-3f08-4102-b75c-ac10382e1959\") " pod="openshift-monitoring/thanos-querier-746f69848d-65tjq" Jul 07 11:12:21.951807 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:21.951680 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:12:22.025769 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:22.025071 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/3e27d653-3f08-4102-b75c-ac10382e1959-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-746f69848d-65tjq\" (UID: \"3e27d653-3f08-4102-b75c-ac10382e1959\") " pod="openshift-monitoring/thanos-querier-746f69848d-65tjq" Jul 07 11:12:22.025769 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:22.025127 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pk65d\" (UniqueName: \"kubernetes.io/projected/3e27d653-3f08-4102-b75c-ac10382e1959-kube-api-access-pk65d\") pod \"thanos-querier-746f69848d-65tjq\" (UID: \"3e27d653-3f08-4102-b75c-ac10382e1959\") " pod="openshift-monitoring/thanos-querier-746f69848d-65tjq" Jul 07 11:12:22.025769 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:22.025168 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3e27d653-3f08-4102-b75c-ac10382e1959-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-746f69848d-65tjq\" (UID: \"3e27d653-3f08-4102-b75c-ac10382e1959\") " pod="openshift-monitoring/thanos-querier-746f69848d-65tjq" Jul 07 11:12:22.025769 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:22.025212 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3e27d653-3f08-4102-b75c-ac10382e1959-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-746f69848d-65tjq\" (UID: \"3e27d653-3f08-4102-b75c-ac10382e1959\") " pod="openshift-monitoring/thanos-querier-746f69848d-65tjq" Jul 07 11:12:22.025769 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:22.025240 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/3e27d653-3f08-4102-b75c-ac10382e1959-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-746f69848d-65tjq\" (UID: \"3e27d653-3f08-4102-b75c-ac10382e1959\") " pod="openshift-monitoring/thanos-querier-746f69848d-65tjq" Jul 07 11:12:22.025769 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:22.025273 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7nvg6\" (UniqueName: \"kubernetes.io/projected/b9ca66a2-4c5a-4816-a081-bb0fecbdab5d-kube-api-access-7nvg6\") pod \"network-check-target-sc5bl\" (UID: \"b9ca66a2-4c5a-4816-a081-bb0fecbdab5d\") " pod="openshift-network-diagnostics/network-check-target-sc5bl" Jul 07 11:12:22.025769 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:22.025309 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3e27d653-3f08-4102-b75c-ac10382e1959-secret-grpc-tls\") pod \"thanos-querier-746f69848d-65tjq\" (UID: \"3e27d653-3f08-4102-b75c-ac10382e1959\") " pod="openshift-monitoring/thanos-querier-746f69848d-65tjq" Jul 07 11:12:22.025769 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:22.025357 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3e27d653-3f08-4102-b75c-ac10382e1959-metrics-client-ca\") pod \"thanos-querier-746f69848d-65tjq\" (UID: \"3e27d653-3f08-4102-b75c-ac10382e1959\") " pod="openshift-monitoring/thanos-querier-746f69848d-65tjq" Jul 07 11:12:22.025769 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:22.025407 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/3e27d653-3f08-4102-b75c-ac10382e1959-secret-thanos-querier-tls\") pod \"thanos-querier-746f69848d-65tjq\" (UID: \"3e27d653-3f08-4102-b75c-ac10382e1959\") " pod="openshift-monitoring/thanos-querier-746f69848d-65tjq" Jul 07 11:12:22.027068 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:22.027002 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3e27d653-3f08-4102-b75c-ac10382e1959-metrics-client-ca\") pod \"thanos-querier-746f69848d-65tjq\" (UID: \"3e27d653-3f08-4102-b75c-ac10382e1959\") " pod="openshift-monitoring/thanos-querier-746f69848d-65tjq" Jul 07 11:12:22.028531 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:22.028504 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/3e27d653-3f08-4102-b75c-ac10382e1959-secret-thanos-querier-tls\") pod \"thanos-querier-746f69848d-65tjq\" (UID: \"3e27d653-3f08-4102-b75c-ac10382e1959\") " pod="openshift-monitoring/thanos-querier-746f69848d-65tjq" Jul 07 11:12:22.028825 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:22.028792 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 07 11:12:22.029015 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:22.028993 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3e27d653-3f08-4102-b75c-ac10382e1959-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-746f69848d-65tjq\" (UID: \"3e27d653-3f08-4102-b75c-ac10382e1959\") " pod="openshift-monitoring/thanos-querier-746f69848d-65tjq" Jul 07 11:12:22.029151 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:22.029132 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/3e27d653-3f08-4102-b75c-ac10382e1959-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-746f69848d-65tjq\" (UID: \"3e27d653-3f08-4102-b75c-ac10382e1959\") " pod="openshift-monitoring/thanos-querier-746f69848d-65tjq" Jul 07 11:12:22.029282 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:22.029262 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3e27d653-3f08-4102-b75c-ac10382e1959-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-746f69848d-65tjq\" (UID: \"3e27d653-3f08-4102-b75c-ac10382e1959\") " pod="openshift-monitoring/thanos-querier-746f69848d-65tjq" Jul 07 11:12:22.029494 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:22.029471 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/3e27d653-3f08-4102-b75c-ac10382e1959-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-746f69848d-65tjq\" (UID: \"3e27d653-3f08-4102-b75c-ac10382e1959\") " pod="openshift-monitoring/thanos-querier-746f69848d-65tjq" Jul 07 11:12:22.029687 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:22.029667 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3e27d653-3f08-4102-b75c-ac10382e1959-secret-grpc-tls\") pod \"thanos-querier-746f69848d-65tjq\" (UID: \"3e27d653-3f08-4102-b75c-ac10382e1959\") " pod="openshift-monitoring/thanos-querier-746f69848d-65tjq" Jul 07 11:12:22.033249 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:22.033228 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-2q46z"] Jul 07 11:12:22.036250 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:12:22.036230 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod94c776b2_5ce9_4210_adf5_31efabcef2af.slice/crio-d2c50151a3137036a386abbde24489b46f010a918c0de731dd5849b842549884 WatchSource:0}: Error finding container d2c50151a3137036a386abbde24489b46f010a918c0de731dd5849b842549884: Status 404 returned error can't find the container with id d2c50151a3137036a386abbde24489b46f010a918c0de731dd5849b842549884 Jul 07 11:12:22.036403 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:22.036386 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pk65d\" (UniqueName: \"kubernetes.io/projected/3e27d653-3f08-4102-b75c-ac10382e1959-kube-api-access-pk65d\") pod \"thanos-querier-746f69848d-65tjq\" (UID: \"3e27d653-3f08-4102-b75c-ac10382e1959\") " pod="openshift-monitoring/thanos-querier-746f69848d-65tjq" Jul 07 11:12:22.038627 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:22.038609 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 07 11:12:22.048403 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:22.048383 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7nvg6\" (UniqueName: \"kubernetes.io/projected/b9ca66a2-4c5a-4816-a081-bb0fecbdab5d-kube-api-access-7nvg6\") pod \"network-check-target-sc5bl\" (UID: \"b9ca66a2-4c5a-4816-a081-bb0fecbdab5d\") " pod="openshift-network-diagnostics/network-check-target-sc5bl" Jul 07 11:12:22.164190 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:22.164147 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-2ffb8" event={"ID":"acd61036-c152-4340-9a8a-cb48eff0f398","Type":"ContainerStarted","Data":"3dfc0d49732355afb9cbfb01d0251d01bbed946807cd5240ef58f8974063184c"} Jul 07 11:12:22.164335 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:22.164206 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-2ffb8" event={"ID":"acd61036-c152-4340-9a8a-cb48eff0f398","Type":"ContainerStarted","Data":"936d1669dc2eb450b52c07fa066f942f728e6e76acaf15bb2fc669ce8891de94"} Jul 07 11:12:22.165475 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:22.165438 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-2q46z" event={"ID":"94c776b2-5ce9-4210-adf5-31efabcef2af","Type":"ContainerStarted","Data":"d2c50151a3137036a386abbde24489b46f010a918c0de731dd5849b842549884"} Jul 07 11:12:22.166247 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:22.166226 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-746f69848d-65tjq" Jul 07 11:12:22.167689 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:22.167665 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-kqpvf" event={"ID":"2ac7717f-0d38-4ac8-b705-d6b9bcbf7859","Type":"ContainerStarted","Data":"24b4a2d00a09b8410cb72120bba9bff4681f44444e8db1c50740d11e6254cf61"} Jul 07 11:12:22.183705 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:22.183659 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-2ffb8" podStartSLOduration=3.193772295 podStartE2EDuration="4.183642954s" totalImagesPullingTime="922.557897ms" totalInitContainerRuntime="67.312762ms" isStatefulPod=true podCreationTimestamp="2026-07-07 11:12:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:12:22.182950694 +0000 UTC m=+66.596819860" watchObservedRunningTime="2026-07-07 11:12:22.183642954 +0000 UTC m=+66.597512119" Jul 07 11:12:22.184726 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:22.184703 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-cn2cx\"" Jul 07 11:12:22.189149 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:22.189120 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:12:22.193215 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:22.193198 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sc5bl" Jul 07 11:12:22.207124 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:22.207084 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-kqpvf" podStartSLOduration=3.025886583 podStartE2EDuration="4.20706711s" totalImagesPullingTime="1.181180527s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:12:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:12:22.206184604 +0000 UTC m=+66.620053774" watchObservedRunningTime="2026-07-07 11:12:22.20706711 +0000 UTC m=+66.620936279" Jul 07 11:12:22.524562 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:22.524535 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-746f69848d-65tjq"] Jul 07 11:12:22.526086 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:12:22.526058 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e27d653_3f08_4102_b75c_ac10382e1959.slice/crio-2051ea8699180aca236d005d576f437d277c667224f6a2adeee42da461582369 WatchSource:0}: Error finding container 2051ea8699180aca236d005d576f437d277c667224f6a2adeee42da461582369: Status 404 returned error can't find the container with id 2051ea8699180aca236d005d576f437d277c667224f6a2adeee42da461582369 Jul 07 11:12:22.537768 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:22.537748 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-sc5bl"] Jul 07 11:12:22.539216 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:12:22.539196 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9ca66a2_4c5a_4816_a081_bb0fecbdab5d.slice/crio-3edfe9bb360d65766f3ca30c41eec2e692d528c3c608c1c039c6e1aa82778fc2 WatchSource:0}: Error finding container 3edfe9bb360d65766f3ca30c41eec2e692d528c3c608c1c039c6e1aa82778fc2: Status 404 returned error can't find the container with id 3edfe9bb360d65766f3ca30c41eec2e692d528c3c608c1c039c6e1aa82778fc2 Jul 07 11:12:23.084115 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:23.084088 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-8654d65bc6-c676s"] Jul 07 11:12:23.086983 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:23.086961 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-8654d65bc6-c676s" Jul 07 11:12:23.091842 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:23.091394 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-875dq\"" Jul 07 11:12:23.091842 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:23.091396 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 07 11:12:23.091842 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:23.091514 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 07 11:12:23.091842 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:23.091677 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 07 11:12:23.091842 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:23.091774 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 07 11:12:23.091842 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:23.091831 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-20qkng69pga46\"" Jul 07 11:12:23.098541 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:23.098521 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-8654d65bc6-c676s"] Jul 07 11:12:23.127894 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:23.127873 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-k8wv9" Jul 07 11:12:23.138837 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:23.138818 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/58c40425-d108-438f-8e91-4af50c22b58c-client-ca-bundle\") pod \"metrics-server-8654d65bc6-c676s\" (UID: \"58c40425-d108-438f-8e91-4af50c22b58c\") " pod="openshift-monitoring/metrics-server-8654d65bc6-c676s" Jul 07 11:12:23.138941 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:23.138868 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/58c40425-d108-438f-8e91-4af50c22b58c-secret-metrics-server-tls\") pod \"metrics-server-8654d65bc6-c676s\" (UID: \"58c40425-d108-438f-8e91-4af50c22b58c\") " pod="openshift-monitoring/metrics-server-8654d65bc6-c676s" Jul 07 11:12:23.138941 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:23.138896 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/58c40425-d108-438f-8e91-4af50c22b58c-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-8654d65bc6-c676s\" (UID: \"58c40425-d108-438f-8e91-4af50c22b58c\") " pod="openshift-monitoring/metrics-server-8654d65bc6-c676s" Jul 07 11:12:23.139041 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:23.138993 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/58c40425-d108-438f-8e91-4af50c22b58c-secret-metrics-server-client-certs\") pod \"metrics-server-8654d65bc6-c676s\" (UID: \"58c40425-d108-438f-8e91-4af50c22b58c\") " pod="openshift-monitoring/metrics-server-8654d65bc6-c676s" Jul 07 11:12:23.139090 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:23.139048 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/58c40425-d108-438f-8e91-4af50c22b58c-metrics-server-audit-profiles\") pod \"metrics-server-8654d65bc6-c676s\" (UID: \"58c40425-d108-438f-8e91-4af50c22b58c\") " pod="openshift-monitoring/metrics-server-8654d65bc6-c676s" Jul 07 11:12:23.139146 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:23.139100 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/58c40425-d108-438f-8e91-4af50c22b58c-audit-log\") pod \"metrics-server-8654d65bc6-c676s\" (UID: \"58c40425-d108-438f-8e91-4af50c22b58c\") " pod="openshift-monitoring/metrics-server-8654d65bc6-c676s" Jul 07 11:12:23.139186 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:23.139172 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-487mh\" (UniqueName: \"kubernetes.io/projected/58c40425-d108-438f-8e91-4af50c22b58c-kube-api-access-487mh\") pod \"metrics-server-8654d65bc6-c676s\" (UID: \"58c40425-d108-438f-8e91-4af50c22b58c\") " pod="openshift-monitoring/metrics-server-8654d65bc6-c676s" Jul 07 11:12:23.175854 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:23.175627 2562 generic.go:356] "Generic (PLEG): container finished" podID="9314229f-a3c5-4809-8129-b06de0b52127" containerID="73d2397143dec700148b876a6451b89628026aae7d4e6f5cb0812bfcf65eb0ce" exitCode=0 Jul 07 11:12:23.175854 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:23.175721 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9314229f-a3c5-4809-8129-b06de0b52127","Type":"ContainerDied","Data":"73d2397143dec700148b876a6451b89628026aae7d4e6f5cb0812bfcf65eb0ce"} Jul 07 11:12:23.178578 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:23.178549 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-2q46z" event={"ID":"94c776b2-5ce9-4210-adf5-31efabcef2af","Type":"ContainerStarted","Data":"aefe3f5c8afb6d8065d71e0dd78db1d947b9a6e9fcb8853018e2d3e44d11ec28"} Jul 07 11:12:23.182271 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:23.182226 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-sc5bl" event={"ID":"b9ca66a2-4c5a-4816-a081-bb0fecbdab5d","Type":"ContainerStarted","Data":"3edfe9bb360d65766f3ca30c41eec2e692d528c3c608c1c039c6e1aa82778fc2"} Jul 07 11:12:23.184394 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:23.184327 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-746f69848d-65tjq" event={"ID":"3e27d653-3f08-4102-b75c-ac10382e1959","Type":"ContainerStarted","Data":"2051ea8699180aca236d005d576f437d277c667224f6a2adeee42da461582369"} Jul 07 11:12:23.239818 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:23.239792 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/58c40425-d108-438f-8e91-4af50c22b58c-metrics-server-audit-profiles\") pod \"metrics-server-8654d65bc6-c676s\" (UID: \"58c40425-d108-438f-8e91-4af50c22b58c\") " pod="openshift-monitoring/metrics-server-8654d65bc6-c676s" Jul 07 11:12:23.239951 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:23.239894 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/58c40425-d108-438f-8e91-4af50c22b58c-audit-log\") pod \"metrics-server-8654d65bc6-c676s\" (UID: \"58c40425-d108-438f-8e91-4af50c22b58c\") " pod="openshift-monitoring/metrics-server-8654d65bc6-c676s" Jul 07 11:12:23.240223 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:23.240197 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-487mh\" (UniqueName: \"kubernetes.io/projected/58c40425-d108-438f-8e91-4af50c22b58c-kube-api-access-487mh\") pod \"metrics-server-8654d65bc6-c676s\" (UID: \"58c40425-d108-438f-8e91-4af50c22b58c\") " pod="openshift-monitoring/metrics-server-8654d65bc6-c676s" Jul 07 11:12:23.240359 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:23.240340 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/58c40425-d108-438f-8e91-4af50c22b58c-client-ca-bundle\") pod \"metrics-server-8654d65bc6-c676s\" (UID: \"58c40425-d108-438f-8e91-4af50c22b58c\") " pod="openshift-monitoring/metrics-server-8654d65bc6-c676s" Jul 07 11:12:23.240478 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:23.240451 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/58c40425-d108-438f-8e91-4af50c22b58c-secret-metrics-server-tls\") pod \"metrics-server-8654d65bc6-c676s\" (UID: \"58c40425-d108-438f-8e91-4af50c22b58c\") " pod="openshift-monitoring/metrics-server-8654d65bc6-c676s" Jul 07 11:12:23.240545 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:23.240496 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/58c40425-d108-438f-8e91-4af50c22b58c-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-8654d65bc6-c676s\" (UID: \"58c40425-d108-438f-8e91-4af50c22b58c\") " pod="openshift-monitoring/metrics-server-8654d65bc6-c676s" Jul 07 11:12:23.241526 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:23.240758 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/58c40425-d108-438f-8e91-4af50c22b58c-secret-metrics-server-client-certs\") pod \"metrics-server-8654d65bc6-c676s\" (UID: \"58c40425-d108-438f-8e91-4af50c22b58c\") " pod="openshift-monitoring/metrics-server-8654d65bc6-c676s" Jul 07 11:12:23.241526 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:23.240831 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/58c40425-d108-438f-8e91-4af50c22b58c-metrics-server-audit-profiles\") pod \"metrics-server-8654d65bc6-c676s\" (UID: \"58c40425-d108-438f-8e91-4af50c22b58c\") " pod="openshift-monitoring/metrics-server-8654d65bc6-c676s" Jul 07 11:12:23.241526 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:23.241214 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/58c40425-d108-438f-8e91-4af50c22b58c-audit-log\") pod \"metrics-server-8654d65bc6-c676s\" (UID: \"58c40425-d108-438f-8e91-4af50c22b58c\") " pod="openshift-monitoring/metrics-server-8654d65bc6-c676s" Jul 07 11:12:23.246486 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:23.243727 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/58c40425-d108-438f-8e91-4af50c22b58c-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-8654d65bc6-c676s\" (UID: \"58c40425-d108-438f-8e91-4af50c22b58c\") " pod="openshift-monitoring/metrics-server-8654d65bc6-c676s" Jul 07 11:12:23.246486 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:23.245405 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/58c40425-d108-438f-8e91-4af50c22b58c-secret-metrics-server-tls\") pod \"metrics-server-8654d65bc6-c676s\" (UID: \"58c40425-d108-438f-8e91-4af50c22b58c\") " pod="openshift-monitoring/metrics-server-8654d65bc6-c676s" Jul 07 11:12:23.256248 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:23.256171 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/58c40425-d108-438f-8e91-4af50c22b58c-client-ca-bundle\") pod \"metrics-server-8654d65bc6-c676s\" (UID: \"58c40425-d108-438f-8e91-4af50c22b58c\") " pod="openshift-monitoring/metrics-server-8654d65bc6-c676s" Jul 07 11:12:23.256825 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:23.256772 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/58c40425-d108-438f-8e91-4af50c22b58c-secret-metrics-server-client-certs\") pod \"metrics-server-8654d65bc6-c676s\" (UID: \"58c40425-d108-438f-8e91-4af50c22b58c\") " pod="openshift-monitoring/metrics-server-8654d65bc6-c676s" Jul 07 11:12:23.259051 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:23.258532 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-487mh\" (UniqueName: \"kubernetes.io/projected/58c40425-d108-438f-8e91-4af50c22b58c-kube-api-access-487mh\") pod \"metrics-server-8654d65bc6-c676s\" (UID: \"58c40425-d108-438f-8e91-4af50c22b58c\") " pod="openshift-monitoring/metrics-server-8654d65bc6-c676s" Jul 07 11:12:23.265278 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:23.265252 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:12:23.366296 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:23.366234 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:12:23.399586 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:23.399562 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-8654d65bc6-c676s" Jul 07 11:12:23.536725 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:23.536695 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-qhxlp"] Jul 07 11:12:23.540478 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:23.540445 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-qhxlp" Jul 07 11:12:23.542797 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:23.542769 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:12:23.542797 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:23.542790 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 07 11:12:23.543023 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:23.542905 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-2cwjt\"" Jul 07 11:12:23.546176 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:23.546142 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-qhxlp"] Jul 07 11:12:23.562524 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:23.562499 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-8654d65bc6-c676s"] Jul 07 11:12:23.565762 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:12:23.565737 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58c40425_d108_438f_8e91_4af50c22b58c.slice/crio-1c95b125605c0a36e0ea35f8d3bcc93340e452907a3db74de012dddb15649dc1 WatchSource:0}: Error finding container 1c95b125605c0a36e0ea35f8d3bcc93340e452907a3db74de012dddb15649dc1: Status 404 returned error can't find the container with id 1c95b125605c0a36e0ea35f8d3bcc93340e452907a3db74de012dddb15649dc1 Jul 07 11:12:23.643672 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:23.643646 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z9c8z\" (UniqueName: \"kubernetes.io/projected/7f341820-bee0-4ac7-964f-4a20ca2eb4f4-kube-api-access-z9c8z\") pod \"monitoring-plugin-d5dcff7cf-qhxlp\" (UID: \"7f341820-bee0-4ac7-964f-4a20ca2eb4f4\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-qhxlp" Jul 07 11:12:23.643784 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:23.643691 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7f341820-bee0-4ac7-964f-4a20ca2eb4f4-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-qhxlp\" (UID: \"7f341820-bee0-4ac7-964f-4a20ca2eb4f4\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-qhxlp" Jul 07 11:12:23.742384 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:23.742345 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:12:23.744412 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:23.744373 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z9c8z\" (UniqueName: \"kubernetes.io/projected/7f341820-bee0-4ac7-964f-4a20ca2eb4f4-kube-api-access-z9c8z\") pod \"monitoring-plugin-d5dcff7cf-qhxlp\" (UID: \"7f341820-bee0-4ac7-964f-4a20ca2eb4f4\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-qhxlp" Jul 07 11:12:23.744546 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:23.744425 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7f341820-bee0-4ac7-964f-4a20ca2eb4f4-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-qhxlp\" (UID: \"7f341820-bee0-4ac7-964f-4a20ca2eb4f4\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-qhxlp" Jul 07 11:12:23.748579 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:23.748532 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7f341820-bee0-4ac7-964f-4a20ca2eb4f4-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-qhxlp\" (UID: \"7f341820-bee0-4ac7-964f-4a20ca2eb4f4\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-qhxlp" Jul 07 11:12:23.760981 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:23.760816 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z9c8z\" (UniqueName: \"kubernetes.io/projected/7f341820-bee0-4ac7-964f-4a20ca2eb4f4-kube-api-access-z9c8z\") pod \"monitoring-plugin-d5dcff7cf-qhxlp\" (UID: \"7f341820-bee0-4ac7-964f-4a20ca2eb4f4\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-qhxlp" Jul 07 11:12:23.854620 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:23.854586 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-qhxlp" Jul 07 11:12:24.001476 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:24.001300 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-5f756946cf-8fnhr"] Jul 07 11:12:24.010035 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:12:24.010005 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f341820_bee0_4ac7_964f_4a20ca2eb4f4.slice/crio-dd5641c15ed97fcb80425ac6aea348b00af60365802016d752204fd56637d427 WatchSource:0}: Error finding container dd5641c15ed97fcb80425ac6aea348b00af60365802016d752204fd56637d427: Status 404 returned error can't find the container with id dd5641c15ed97fcb80425ac6aea348b00af60365802016d752204fd56637d427 Jul 07 11:12:24.021851 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:24.021825 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-qhxlp"] Jul 07 11:12:24.021985 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:24.021950 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-5f756946cf-8fnhr" Jul 07 11:12:24.023298 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:24.021952 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-5f756946cf-8fnhr"] Jul 07 11:12:24.025083 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:24.024836 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 07 11:12:24.025083 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:24.024966 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-k55xg\"" Jul 07 11:12:24.025285 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:24.025161 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 07 11:12:24.025715 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:24.025505 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 07 11:12:24.025715 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:24.025693 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 07 11:12:24.026100 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:24.025955 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 07 11:12:24.032896 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:24.032710 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 07 11:12:24.149878 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:24.149841 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0eea7695-f8a3-4f8e-aea5-f619a6e06fcb-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5f756946cf-8fnhr\" (UID: \"0eea7695-f8a3-4f8e-aea5-f619a6e06fcb\") " pod="openshift-monitoring/telemeter-client-5f756946cf-8fnhr" Jul 07 11:12:24.149878 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:24.149880 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0eea7695-f8a3-4f8e-aea5-f619a6e06fcb-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5f756946cf-8fnhr\" (UID: \"0eea7695-f8a3-4f8e-aea5-f619a6e06fcb\") " pod="openshift-monitoring/telemeter-client-5f756946cf-8fnhr" Jul 07 11:12:24.150304 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:24.149933 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0eea7695-f8a3-4f8e-aea5-f619a6e06fcb-metrics-client-ca\") pod \"telemeter-client-5f756946cf-8fnhr\" (UID: \"0eea7695-f8a3-4f8e-aea5-f619a6e06fcb\") " pod="openshift-monitoring/telemeter-client-5f756946cf-8fnhr" Jul 07 11:12:24.150304 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:24.150042 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0eea7695-f8a3-4f8e-aea5-f619a6e06fcb-serving-certs-ca-bundle\") pod \"telemeter-client-5f756946cf-8fnhr\" (UID: \"0eea7695-f8a3-4f8e-aea5-f619a6e06fcb\") " pod="openshift-monitoring/telemeter-client-5f756946cf-8fnhr" Jul 07 11:12:24.150304 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:24.150076 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/0eea7695-f8a3-4f8e-aea5-f619a6e06fcb-telemeter-client-tls\") pod \"telemeter-client-5f756946cf-8fnhr\" (UID: \"0eea7695-f8a3-4f8e-aea5-f619a6e06fcb\") " pod="openshift-monitoring/telemeter-client-5f756946cf-8fnhr" Jul 07 11:12:24.150304 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:24.150122 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/0eea7695-f8a3-4f8e-aea5-f619a6e06fcb-federate-client-tls\") pod \"telemeter-client-5f756946cf-8fnhr\" (UID: \"0eea7695-f8a3-4f8e-aea5-f619a6e06fcb\") " pod="openshift-monitoring/telemeter-client-5f756946cf-8fnhr" Jul 07 11:12:24.150304 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:24.150206 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k6jvf\" (UniqueName: \"kubernetes.io/projected/0eea7695-f8a3-4f8e-aea5-f619a6e06fcb-kube-api-access-k6jvf\") pod \"telemeter-client-5f756946cf-8fnhr\" (UID: \"0eea7695-f8a3-4f8e-aea5-f619a6e06fcb\") " pod="openshift-monitoring/telemeter-client-5f756946cf-8fnhr" Jul 07 11:12:24.150304 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:24.150232 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/0eea7695-f8a3-4f8e-aea5-f619a6e06fcb-secret-telemeter-client\") pod \"telemeter-client-5f756946cf-8fnhr\" (UID: \"0eea7695-f8a3-4f8e-aea5-f619a6e06fcb\") " pod="openshift-monitoring/telemeter-client-5f756946cf-8fnhr" Jul 07 11:12:24.189840 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:24.189713 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-qhxlp" event={"ID":"7f341820-bee0-4ac7-964f-4a20ca2eb4f4","Type":"ContainerStarted","Data":"dd5641c15ed97fcb80425ac6aea348b00af60365802016d752204fd56637d427"} Jul 07 11:12:24.191237 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:24.191209 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-8654d65bc6-c676s" event={"ID":"58c40425-d108-438f-8e91-4af50c22b58c","Type":"ContainerStarted","Data":"1c95b125605c0a36e0ea35f8d3bcc93340e452907a3db74de012dddb15649dc1"} Jul 07 11:12:24.195202 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:24.195173 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-2q46z" event={"ID":"94c776b2-5ce9-4210-adf5-31efabcef2af","Type":"ContainerStarted","Data":"1d40891d481e618899fba9602a7949a50fd6d68a3d5a5afa6fdc03e6a925ac0f"} Jul 07 11:12:24.250948 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:24.250897 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/0eea7695-f8a3-4f8e-aea5-f619a6e06fcb-federate-client-tls\") pod \"telemeter-client-5f756946cf-8fnhr\" (UID: \"0eea7695-f8a3-4f8e-aea5-f619a6e06fcb\") " pod="openshift-monitoring/telemeter-client-5f756946cf-8fnhr" Jul 07 11:12:24.251080 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:24.251008 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k6jvf\" (UniqueName: \"kubernetes.io/projected/0eea7695-f8a3-4f8e-aea5-f619a6e06fcb-kube-api-access-k6jvf\") pod \"telemeter-client-5f756946cf-8fnhr\" (UID: \"0eea7695-f8a3-4f8e-aea5-f619a6e06fcb\") " pod="openshift-monitoring/telemeter-client-5f756946cf-8fnhr" Jul 07 11:12:24.251080 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:24.251063 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/0eea7695-f8a3-4f8e-aea5-f619a6e06fcb-secret-telemeter-client\") pod \"telemeter-client-5f756946cf-8fnhr\" (UID: \"0eea7695-f8a3-4f8e-aea5-f619a6e06fcb\") " pod="openshift-monitoring/telemeter-client-5f756946cf-8fnhr" Jul 07 11:12:24.251201 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:24.251108 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0eea7695-f8a3-4f8e-aea5-f619a6e06fcb-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5f756946cf-8fnhr\" (UID: \"0eea7695-f8a3-4f8e-aea5-f619a6e06fcb\") " pod="openshift-monitoring/telemeter-client-5f756946cf-8fnhr" Jul 07 11:12:24.251201 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:24.251139 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0eea7695-f8a3-4f8e-aea5-f619a6e06fcb-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5f756946cf-8fnhr\" (UID: \"0eea7695-f8a3-4f8e-aea5-f619a6e06fcb\") " pod="openshift-monitoring/telemeter-client-5f756946cf-8fnhr" Jul 07 11:12:24.251201 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:24.251182 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0eea7695-f8a3-4f8e-aea5-f619a6e06fcb-metrics-client-ca\") pod \"telemeter-client-5f756946cf-8fnhr\" (UID: \"0eea7695-f8a3-4f8e-aea5-f619a6e06fcb\") " pod="openshift-monitoring/telemeter-client-5f756946cf-8fnhr" Jul 07 11:12:24.251357 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:24.251224 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0eea7695-f8a3-4f8e-aea5-f619a6e06fcb-serving-certs-ca-bundle\") pod \"telemeter-client-5f756946cf-8fnhr\" (UID: \"0eea7695-f8a3-4f8e-aea5-f619a6e06fcb\") " pod="openshift-monitoring/telemeter-client-5f756946cf-8fnhr" Jul 07 11:12:24.251357 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:24.251253 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/0eea7695-f8a3-4f8e-aea5-f619a6e06fcb-telemeter-client-tls\") pod \"telemeter-client-5f756946cf-8fnhr\" (UID: \"0eea7695-f8a3-4f8e-aea5-f619a6e06fcb\") " pod="openshift-monitoring/telemeter-client-5f756946cf-8fnhr" Jul 07 11:12:24.253847 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:24.253818 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0eea7695-f8a3-4f8e-aea5-f619a6e06fcb-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5f756946cf-8fnhr\" (UID: \"0eea7695-f8a3-4f8e-aea5-f619a6e06fcb\") " pod="openshift-monitoring/telemeter-client-5f756946cf-8fnhr" Jul 07 11:12:24.253983 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:24.253887 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0eea7695-f8a3-4f8e-aea5-f619a6e06fcb-metrics-client-ca\") pod \"telemeter-client-5f756946cf-8fnhr\" (UID: \"0eea7695-f8a3-4f8e-aea5-f619a6e06fcb\") " pod="openshift-monitoring/telemeter-client-5f756946cf-8fnhr" Jul 07 11:12:24.254459 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:24.254433 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0eea7695-f8a3-4f8e-aea5-f619a6e06fcb-serving-certs-ca-bundle\") pod \"telemeter-client-5f756946cf-8fnhr\" (UID: \"0eea7695-f8a3-4f8e-aea5-f619a6e06fcb\") " pod="openshift-monitoring/telemeter-client-5f756946cf-8fnhr" Jul 07 11:12:24.257864 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:24.257808 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0eea7695-f8a3-4f8e-aea5-f619a6e06fcb-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5f756946cf-8fnhr\" (UID: \"0eea7695-f8a3-4f8e-aea5-f619a6e06fcb\") " pod="openshift-monitoring/telemeter-client-5f756946cf-8fnhr" Jul 07 11:12:24.263401 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:24.263349 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/0eea7695-f8a3-4f8e-aea5-f619a6e06fcb-secret-telemeter-client\") pod \"telemeter-client-5f756946cf-8fnhr\" (UID: \"0eea7695-f8a3-4f8e-aea5-f619a6e06fcb\") " pod="openshift-monitoring/telemeter-client-5f756946cf-8fnhr" Jul 07 11:12:24.264267 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:24.263995 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/0eea7695-f8a3-4f8e-aea5-f619a6e06fcb-telemeter-client-tls\") pod \"telemeter-client-5f756946cf-8fnhr\" (UID: \"0eea7695-f8a3-4f8e-aea5-f619a6e06fcb\") " pod="openshift-monitoring/telemeter-client-5f756946cf-8fnhr" Jul 07 11:12:24.268499 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:24.268447 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/0eea7695-f8a3-4f8e-aea5-f619a6e06fcb-federate-client-tls\") pod \"telemeter-client-5f756946cf-8fnhr\" (UID: \"0eea7695-f8a3-4f8e-aea5-f619a6e06fcb\") " pod="openshift-monitoring/telemeter-client-5f756946cf-8fnhr" Jul 07 11:12:24.274014 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:24.273990 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k6jvf\" (UniqueName: \"kubernetes.io/projected/0eea7695-f8a3-4f8e-aea5-f619a6e06fcb-kube-api-access-k6jvf\") pod \"telemeter-client-5f756946cf-8fnhr\" (UID: \"0eea7695-f8a3-4f8e-aea5-f619a6e06fcb\") " pod="openshift-monitoring/telemeter-client-5f756946cf-8fnhr" Jul 07 11:12:24.336988 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:24.336959 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-5f756946cf-8fnhr" Jul 07 11:12:24.343974 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:24.343944 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:12:24.445383 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:24.445294 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:12:24.584598 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:24.584273 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-2q46z" podStartSLOduration=65.578321353 podStartE2EDuration="1m6.584249121s" totalImagesPullingTime="1.005927768s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:11:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:12:24.213430631 +0000 UTC m=+68.627299808" watchObservedRunningTime="2026-07-07 11:12:24.584249121 +0000 UTC m=+68.998118287" Jul 07 11:12:24.586573 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:24.585197 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-685bfb5f64-428ms"] Jul 07 11:12:24.588421 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:24.587909 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-685bfb5f64-428ms" Jul 07 11:12:24.590637 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:24.590616 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-ttmcq\"" Jul 07 11:12:24.590973 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:24.590942 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 07 11:12:24.591078 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:24.590995 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 07 11:12:24.612676 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:24.612599 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:12:24.625559 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:24.625500 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-685bfb5f64-428ms"] Jul 07 11:12:24.655159 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:24.655105 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7vbbv\" (UniqueName: \"kubernetes.io/projected/a40bc7e6-49a7-49f7-b204-e03679e7f1fc-kube-api-access-7vbbv\") pod \"downloads-685bfb5f64-428ms\" (UID: \"a40bc7e6-49a7-49f7-b204-e03679e7f1fc\") " pod="openshift-console/downloads-685bfb5f64-428ms" Jul 07 11:12:24.723540 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:24.723468 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:12:24.756256 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:24.756228 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7vbbv\" (UniqueName: \"kubernetes.io/projected/a40bc7e6-49a7-49f7-b204-e03679e7f1fc-kube-api-access-7vbbv\") pod \"downloads-685bfb5f64-428ms\" (UID: \"a40bc7e6-49a7-49f7-b204-e03679e7f1fc\") " pod="openshift-console/downloads-685bfb5f64-428ms" Jul 07 11:12:24.765901 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:24.765847 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7vbbv\" (UniqueName: \"kubernetes.io/projected/a40bc7e6-49a7-49f7-b204-e03679e7f1fc-kube-api-access-7vbbv\") pod \"downloads-685bfb5f64-428ms\" (UID: \"a40bc7e6-49a7-49f7-b204-e03679e7f1fc\") " pod="openshift-console/downloads-685bfb5f64-428ms" Jul 07 11:12:24.901972 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:24.901871 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-685bfb5f64-428ms" Jul 07 11:12:24.924826 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:24.924761 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:12:25.087991 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:25.087801 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:12:25.094653 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:25.092765 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 07 11:12:25.096972 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:25.096950 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:12:25.108946 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:25.103082 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 07 11:12:25.108946 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:25.104039 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 07 11:12:25.108946 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:25.104335 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 07 11:12:25.108946 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:25.104353 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 07 11:12:25.108946 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:25.105042 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 07 11:12:25.108946 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:25.105579 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 07 11:12:25.108946 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:25.105879 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 07 11:12:25.108946 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:25.106615 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-9jb136q3vmpab\"" Jul 07 11:12:25.108946 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:25.106645 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 07 11:12:25.108946 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:25.107002 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 07 11:12:25.108946 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:25.107359 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 07 11:12:25.108946 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:25.107609 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-nln8s\"" Jul 07 11:12:25.108946 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:25.107847 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 07 11:12:25.109633 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:25.109338 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 07 11:12:25.114990 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:25.114970 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 07 11:12:25.121032 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:25.121009 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 07 11:12:25.121298 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:25.121273 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 07 11:12:25.159013 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:25.158980 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c088023f-86df-4531-be68-1a21bad47ae6-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c088023f-86df-4531-be68-1a21bad47ae6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:12:25.159443 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:25.159027 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c088023f-86df-4531-be68-1a21bad47ae6-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"c088023f-86df-4531-be68-1a21bad47ae6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:12:25.159443 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:25.159065 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/c088023f-86df-4531-be68-1a21bad47ae6-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"c088023f-86df-4531-be68-1a21bad47ae6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:12:25.159443 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:25.159103 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c088023f-86df-4531-be68-1a21bad47ae6-config-out\") pod \"prometheus-k8s-0\" (UID: \"c088023f-86df-4531-be68-1a21bad47ae6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:12:25.159443 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:25.159136 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/c088023f-86df-4531-be68-1a21bad47ae6-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"c088023f-86df-4531-be68-1a21bad47ae6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:12:25.159443 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:25.159170 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/c088023f-86df-4531-be68-1a21bad47ae6-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"c088023f-86df-4531-be68-1a21bad47ae6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:12:25.159443 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:25.159199 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c088023f-86df-4531-be68-1a21bad47ae6-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"c088023f-86df-4531-be68-1a21bad47ae6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:12:25.159443 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:25.159222 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c088023f-86df-4531-be68-1a21bad47ae6-web-config\") pod \"prometheus-k8s-0\" (UID: \"c088023f-86df-4531-be68-1a21bad47ae6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:12:25.159443 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:25.159263 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/c088023f-86df-4531-be68-1a21bad47ae6-config\") pod \"prometheus-k8s-0\" (UID: \"c088023f-86df-4531-be68-1a21bad47ae6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:12:25.159443 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:25.159287 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/c088023f-86df-4531-be68-1a21bad47ae6-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"c088023f-86df-4531-be68-1a21bad47ae6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:12:25.159443 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:25.159313 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-npr2b\" (UniqueName: \"kubernetes.io/projected/c088023f-86df-4531-be68-1a21bad47ae6-kube-api-access-npr2b\") pod \"prometheus-k8s-0\" (UID: \"c088023f-86df-4531-be68-1a21bad47ae6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:12:25.159443 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:25.159376 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/c088023f-86df-4531-be68-1a21bad47ae6-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"c088023f-86df-4531-be68-1a21bad47ae6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:12:25.159443 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:25.159419 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c088023f-86df-4531-be68-1a21bad47ae6-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"c088023f-86df-4531-be68-1a21bad47ae6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:12:25.159443 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:25.159446 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/c088023f-86df-4531-be68-1a21bad47ae6-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"c088023f-86df-4531-be68-1a21bad47ae6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:12:25.160112 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:25.159505 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c088023f-86df-4531-be68-1a21bad47ae6-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"c088023f-86df-4531-be68-1a21bad47ae6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:12:25.160112 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:25.159538 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/c088023f-86df-4531-be68-1a21bad47ae6-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"c088023f-86df-4531-be68-1a21bad47ae6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:12:25.160112 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:25.159578 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c088023f-86df-4531-be68-1a21bad47ae6-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"c088023f-86df-4531-be68-1a21bad47ae6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:12:25.160112 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:25.159603 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/c088023f-86df-4531-be68-1a21bad47ae6-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"c088023f-86df-4531-be68-1a21bad47ae6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:12:25.160112 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:25.159645 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c088023f-86df-4531-be68-1a21bad47ae6-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c088023f-86df-4531-be68-1a21bad47ae6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:12:25.160112 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:25.159677 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c088023f-86df-4531-be68-1a21bad47ae6-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c088023f-86df-4531-be68-1a21bad47ae6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:12:25.260777 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:25.260744 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/c088023f-86df-4531-be68-1a21bad47ae6-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"c088023f-86df-4531-be68-1a21bad47ae6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:12:25.260978 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:25.260804 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c088023f-86df-4531-be68-1a21bad47ae6-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"c088023f-86df-4531-be68-1a21bad47ae6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:12:25.260978 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:25.260835 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/c088023f-86df-4531-be68-1a21bad47ae6-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"c088023f-86df-4531-be68-1a21bad47ae6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:12:25.260978 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:25.260885 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c088023f-86df-4531-be68-1a21bad47ae6-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"c088023f-86df-4531-be68-1a21bad47ae6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:12:25.260978 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:25.260931 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/c088023f-86df-4531-be68-1a21bad47ae6-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"c088023f-86df-4531-be68-1a21bad47ae6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:12:25.260978 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:25.260966 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c088023f-86df-4531-be68-1a21bad47ae6-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"c088023f-86df-4531-be68-1a21bad47ae6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:12:25.261248 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:25.260994 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/c088023f-86df-4531-be68-1a21bad47ae6-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"c088023f-86df-4531-be68-1a21bad47ae6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:12:25.261248 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:25.261032 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c088023f-86df-4531-be68-1a21bad47ae6-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c088023f-86df-4531-be68-1a21bad47ae6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:12:25.261248 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:25.261062 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c088023f-86df-4531-be68-1a21bad47ae6-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c088023f-86df-4531-be68-1a21bad47ae6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:12:25.261248 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:25.261102 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c088023f-86df-4531-be68-1a21bad47ae6-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c088023f-86df-4531-be68-1a21bad47ae6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:12:25.261248 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:25.261130 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c088023f-86df-4531-be68-1a21bad47ae6-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"c088023f-86df-4531-be68-1a21bad47ae6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:12:25.261248 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:25.261207 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/c088023f-86df-4531-be68-1a21bad47ae6-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"c088023f-86df-4531-be68-1a21bad47ae6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:12:25.261248 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:25.261247 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c088023f-86df-4531-be68-1a21bad47ae6-config-out\") pod \"prometheus-k8s-0\" (UID: \"c088023f-86df-4531-be68-1a21bad47ae6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:12:25.261635 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:25.261282 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/c088023f-86df-4531-be68-1a21bad47ae6-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"c088023f-86df-4531-be68-1a21bad47ae6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:12:25.261635 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:25.261317 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/c088023f-86df-4531-be68-1a21bad47ae6-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"c088023f-86df-4531-be68-1a21bad47ae6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:12:25.261635 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:25.261347 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c088023f-86df-4531-be68-1a21bad47ae6-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"c088023f-86df-4531-be68-1a21bad47ae6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:12:25.261635 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:25.261373 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c088023f-86df-4531-be68-1a21bad47ae6-web-config\") pod \"prometheus-k8s-0\" (UID: \"c088023f-86df-4531-be68-1a21bad47ae6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:12:25.261635 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:25.261433 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/c088023f-86df-4531-be68-1a21bad47ae6-config\") pod \"prometheus-k8s-0\" (UID: \"c088023f-86df-4531-be68-1a21bad47ae6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:12:25.261635 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:25.261457 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/c088023f-86df-4531-be68-1a21bad47ae6-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"c088023f-86df-4531-be68-1a21bad47ae6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:12:25.261635 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:25.261482 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-npr2b\" (UniqueName: \"kubernetes.io/projected/c088023f-86df-4531-be68-1a21bad47ae6-kube-api-access-npr2b\") pod \"prometheus-k8s-0\" (UID: \"c088023f-86df-4531-be68-1a21bad47ae6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:12:25.262583 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:25.262296 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c088023f-86df-4531-be68-1a21bad47ae6-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c088023f-86df-4531-be68-1a21bad47ae6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:12:25.266203 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:25.265643 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/c088023f-86df-4531-be68-1a21bad47ae6-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"c088023f-86df-4531-be68-1a21bad47ae6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:12:25.267617 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:25.267404 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/c088023f-86df-4531-be68-1a21bad47ae6-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"c088023f-86df-4531-be68-1a21bad47ae6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:12:25.267617 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:25.267511 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/c088023f-86df-4531-be68-1a21bad47ae6-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"c088023f-86df-4531-be68-1a21bad47ae6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:12:25.267775 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:25.267652 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c088023f-86df-4531-be68-1a21bad47ae6-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c088023f-86df-4531-be68-1a21bad47ae6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:12:25.268640 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:25.268195 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/c088023f-86df-4531-be68-1a21bad47ae6-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"c088023f-86df-4531-be68-1a21bad47ae6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:12:25.268640 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:25.268209 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c088023f-86df-4531-be68-1a21bad47ae6-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c088023f-86df-4531-be68-1a21bad47ae6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:12:25.268640 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:25.268601 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/c088023f-86df-4531-be68-1a21bad47ae6-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"c088023f-86df-4531-be68-1a21bad47ae6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:12:25.268831 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:25.268647 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c088023f-86df-4531-be68-1a21bad47ae6-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"c088023f-86df-4531-be68-1a21bad47ae6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:12:25.269600 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:25.269576 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/c088023f-86df-4531-be68-1a21bad47ae6-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"c088023f-86df-4531-be68-1a21bad47ae6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:12:25.270330 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:25.270288 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/c088023f-86df-4531-be68-1a21bad47ae6-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"c088023f-86df-4531-be68-1a21bad47ae6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:12:25.270410 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:25.270351 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c088023f-86df-4531-be68-1a21bad47ae6-config-out\") pod \"prometheus-k8s-0\" (UID: \"c088023f-86df-4531-be68-1a21bad47ae6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:12:25.272106 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:25.271672 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c088023f-86df-4531-be68-1a21bad47ae6-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"c088023f-86df-4531-be68-1a21bad47ae6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:12:25.272106 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:25.271802 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c088023f-86df-4531-be68-1a21bad47ae6-web-config\") pod \"prometheus-k8s-0\" (UID: \"c088023f-86df-4531-be68-1a21bad47ae6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:12:25.272106 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:25.272063 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c088023f-86df-4531-be68-1a21bad47ae6-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"c088023f-86df-4531-be68-1a21bad47ae6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:12:25.272780 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:25.272742 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/c088023f-86df-4531-be68-1a21bad47ae6-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"c088023f-86df-4531-be68-1a21bad47ae6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:12:25.273804 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:25.273749 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-npr2b\" (UniqueName: \"kubernetes.io/projected/c088023f-86df-4531-be68-1a21bad47ae6-kube-api-access-npr2b\") pod \"prometheus-k8s-0\" (UID: \"c088023f-86df-4531-be68-1a21bad47ae6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:12:25.274180 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:25.274159 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c088023f-86df-4531-be68-1a21bad47ae6-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"c088023f-86df-4531-be68-1a21bad47ae6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:12:25.279167 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:25.279142 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c088023f-86df-4531-be68-1a21bad47ae6-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"c088023f-86df-4531-be68-1a21bad47ae6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:12:25.280245 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:25.280204 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/c088023f-86df-4531-be68-1a21bad47ae6-config\") pod \"prometheus-k8s-0\" (UID: \"c088023f-86df-4531-be68-1a21bad47ae6\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:12:25.350776 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:25.350694 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:12:25.420105 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:25.420071 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:12:25.427311 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:25.427051 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:12:26.473096 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:26.473064 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:12:26.679305 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:26.679271 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:12:26.779889 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:26.779803 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:12:26.847780 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:26.847749 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:12:27.372030 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:27.371977 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-5f756946cf-8fnhr"] Jul 07 11:12:27.379290 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:12:27.379244 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0eea7695_f8a3_4f8e_aea5_f619a6e06fcb.slice/crio-6eda206f099bd535e725f86a516dd42754cd88b1f1dd9b342e83e359c7baa933 WatchSource:0}: Error finding container 6eda206f099bd535e725f86a516dd42754cd88b1f1dd9b342e83e359c7baa933: Status 404 returned error can't find the container with id 6eda206f099bd535e725f86a516dd42754cd88b1f1dd9b342e83e359c7baa933 Jul 07 11:12:27.389277 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:27.389214 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-685bfb5f64-428ms"] Jul 07 11:12:27.411504 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:27.411485 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 07 11:12:27.414146 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:12:27.414126 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc088023f_86df_4531_be68_1a21bad47ae6.slice/crio-0c75b9c0bb1bcfd0f25e44c6d258242900fc2539c7fb56e3597df424f567be22 WatchSource:0}: Error finding container 0c75b9c0bb1bcfd0f25e44c6d258242900fc2539c7fb56e3597df424f567be22: Status 404 returned error can't find the container with id 0c75b9c0bb1bcfd0f25e44c6d258242900fc2539c7fb56e3597df424f567be22 Jul 07 11:12:28.250462 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:28.250418 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9314229f-a3c5-4809-8129-b06de0b52127","Type":"ContainerStarted","Data":"fda1eb6c9a21a1a11d784870ce053940d72f5040e7cfe866003bb2a64b4a8ed3"} Jul 07 11:12:28.250957 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:28.250534 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9314229f-a3c5-4809-8129-b06de0b52127","Type":"ContainerStarted","Data":"a837483aac629f55a9de413741ae4ac7ac5c5b47f1015ee044a7a2845388f98f"} Jul 07 11:12:28.250957 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:28.250583 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9314229f-a3c5-4809-8129-b06de0b52127","Type":"ContainerStarted","Data":"6ecec6f2bf77b54f5f6df144e16b23b8ddd9fa14e01930a3ee23841f970a5fc2"} Jul 07 11:12:28.250957 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:28.250629 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9314229f-a3c5-4809-8129-b06de0b52127","Type":"ContainerStarted","Data":"0dbb621e5297345bd37afa912a1dcdeb365469f041479464d014b99beb6245af"} Jul 07 11:12:28.250957 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:28.250666 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9314229f-a3c5-4809-8129-b06de0b52127","Type":"ContainerStarted","Data":"1852311dd0b3159a0c90baf81ae46cf25809a80ec728f190fc163a7dfae2a336"} Jul 07 11:12:28.252532 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:28.252504 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-sc5bl" event={"ID":"b9ca66a2-4c5a-4816-a081-bb0fecbdab5d","Type":"ContainerStarted","Data":"9b29a9a0b804491a3029d531c96816953a63006279e03b3445f53e300429abe9"} Jul 07 11:12:28.252665 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:28.252591 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-sc5bl" Jul 07 11:12:28.254400 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:28.254373 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-qhxlp" event={"ID":"7f341820-bee0-4ac7-964f-4a20ca2eb4f4","Type":"ContainerStarted","Data":"12aa0609e44222c47d020e429ef284c6dc7dee2b373f1b726448d29a79c1655e"} Jul 07 11:12:28.254525 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:28.254485 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-qhxlp" Jul 07 11:12:28.256226 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:28.256204 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-8654d65bc6-c676s" event={"ID":"58c40425-d108-438f-8e91-4af50c22b58c","Type":"ContainerStarted","Data":"76efac4c4545db16515bce38453db17fadd3530c510e1948cd2d72c655e1040f"} Jul 07 11:12:28.258337 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:28.258310 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5f756946cf-8fnhr" event={"ID":"0eea7695-f8a3-4f8e-aea5-f619a6e06fcb","Type":"ContainerStarted","Data":"6eda206f099bd535e725f86a516dd42754cd88b1f1dd9b342e83e359c7baa933"} Jul 07 11:12:28.262684 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:28.262467 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-746f69848d-65tjq" event={"ID":"3e27d653-3f08-4102-b75c-ac10382e1959","Type":"ContainerStarted","Data":"7f8f65daf726addf00b8a447f43047c2715dded2e010bfad79f27e515513b624"} Jul 07 11:12:28.262684 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:28.262563 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-qhxlp" Jul 07 11:12:28.262684 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:28.262589 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-746f69848d-65tjq" event={"ID":"3e27d653-3f08-4102-b75c-ac10382e1959","Type":"ContainerStarted","Data":"b512ae5d55299fe5f9886d68e2f3813b5c8a005344c0898f5e4095a17f0dadcc"} Jul 07 11:12:28.262684 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:28.262635 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-746f69848d-65tjq" event={"ID":"3e27d653-3f08-4102-b75c-ac10382e1959","Type":"ContainerStarted","Data":"c46b5780828fa0abd087ad26b32269df6add91254906eaf71f4a0543148f2953"} Jul 07 11:12:28.264613 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:28.264565 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-685bfb5f64-428ms" event={"ID":"a40bc7e6-49a7-49f7-b204-e03679e7f1fc","Type":"ContainerStarted","Data":"56462cb9a86d66153299207eaff95d27668ee31e42fa9a731fa983abde5b0942"} Jul 07 11:12:28.267111 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:28.267076 2562 generic.go:356] "Generic (PLEG): container finished" podID="c088023f-86df-4531-be68-1a21bad47ae6" containerID="540adf46eb218874649adb7703958545843d4594f01f897d43762bc2cf395532" exitCode=0 Jul 07 11:12:28.267222 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:28.267144 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c088023f-86df-4531-be68-1a21bad47ae6","Type":"ContainerDied","Data":"540adf46eb218874649adb7703958545843d4594f01f897d43762bc2cf395532"} Jul 07 11:12:28.267222 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:28.267206 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c088023f-86df-4531-be68-1a21bad47ae6","Type":"ContainerStarted","Data":"0c75b9c0bb1bcfd0f25e44c6d258242900fc2539c7fb56e3597df424f567be22"} Jul 07 11:12:28.288936 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:28.288735 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-sc5bl" podStartSLOduration=65.624914924 podStartE2EDuration="1m10.288716574s" totalImagesPullingTime="4.66380165s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:11:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:12:28.26814653 +0000 UTC m=+72.682015695" watchObservedRunningTime="2026-07-07 11:12:28.288716574 +0000 UTC m=+72.702585743" Jul 07 11:12:28.308441 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:28.308390 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-8654d65bc6-c676s" podStartSLOduration=1.703620605 podStartE2EDuration="5.308371559s" totalImagesPullingTime="3.604750954s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:12:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:12:28.288680482 +0000 UTC m=+72.702549649" watchObservedRunningTime="2026-07-07 11:12:28.308371559 +0000 UTC m=+72.722240724" Jul 07 11:12:28.309076 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:28.309024 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-qhxlp" podStartSLOduration=2.145481603 podStartE2EDuration="5.30900897s" totalImagesPullingTime="3.163527367s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:12:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:12:28.30739871 +0000 UTC m=+72.721267875" watchObservedRunningTime="2026-07-07 11:12:28.30900897 +0000 UTC m=+72.722878136" Jul 07 11:12:28.605866 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:28.605827 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:12:28.662195 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:28.662158 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:12:28.767115 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:28.767005 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:12:28.822370 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:28.822337 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:12:29.278396 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:29.277686 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9314229f-a3c5-4809-8129-b06de0b52127","Type":"ContainerStarted","Data":"49442c22eefabdd95e299db5a21b5eae8a7f2cbb6806da0aa00fba206790df3f"} Jul 07 11:12:29.282513 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:29.282390 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-746f69848d-65tjq" event={"ID":"3e27d653-3f08-4102-b75c-ac10382e1959","Type":"ContainerStarted","Data":"b695ce164c3694716cd81fd19271d713218f2b6e2d7a6d82af7aed3909c296ef"} Jul 07 11:12:29.282513 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:29.282477 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-746f69848d-65tjq" event={"ID":"3e27d653-3f08-4102-b75c-ac10382e1959","Type":"ContainerStarted","Data":"6dba631a67c6a428297ac29a10e37815f954dd975b9ce1cf94e9453fdbd22cf7"} Jul 07 11:12:29.282716 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:29.282527 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-746f69848d-65tjq" event={"ID":"3e27d653-3f08-4102-b75c-ac10382e1959","Type":"ContainerStarted","Data":"c567048fedd15cb25f518953fc89a79ce1ff2ed87396f5c63e6a9d417b6828cb"} Jul 07 11:12:29.311629 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:29.311572 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=4.221089056 podStartE2EDuration="10.311551804s" totalImagesPullingTime="6.072036855s" totalInitContainerRuntime="18.425893ms" isStatefulPod=false podCreationTimestamp="2026-07-07 11:12:19 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:12:29.309411718 +0000 UTC m=+73.723280884" watchObservedRunningTime="2026-07-07 11:12:29.311551804 +0000 UTC m=+73.725420970" Jul 07 11:12:29.340253 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:29.340190 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-746f69848d-65tjq" podStartSLOduration=2.9172644549999998 podStartE2EDuration="8.340169194s" totalImagesPullingTime="5.422904739s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:12:21 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:12:29.337893299 +0000 UTC m=+73.751762464" watchObservedRunningTime="2026-07-07 11:12:29.340169194 +0000 UTC m=+73.754038360" Jul 07 11:12:30.187708 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:30.187661 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:12:30.290982 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:30.290885 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:12:30.294572 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:30.294545 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5f756946cf-8fnhr" event={"ID":"0eea7695-f8a3-4f8e-aea5-f619a6e06fcb","Type":"ContainerStarted","Data":"564cc68567f78931ba4bc51d6eb9a13ccf9859c0a6e60f6bed246d7eb3946c08"} Jul 07 11:12:30.294689 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:30.294615 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5f756946cf-8fnhr" event={"ID":"0eea7695-f8a3-4f8e-aea5-f619a6e06fcb","Type":"ContainerStarted","Data":"cb87e5ad2ec7d6c9c93088fccabf358104928e5f2d58013d9f20a3b6d6bd1160"} Jul 07 11:12:30.294689 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:30.294645 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5f756946cf-8fnhr" event={"ID":"0eea7695-f8a3-4f8e-aea5-f619a6e06fcb","Type":"ContainerStarted","Data":"9e806fccb6a0382f5a1dcea108633faffb58ebd06a602f1c4a1cb7b94074d3c2"} Jul 07 11:12:30.296228 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:30.296209 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-746f69848d-65tjq" Jul 07 11:12:30.323684 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:30.323495 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-5f756946cf-8fnhr" podStartSLOduration=4.872455227 podStartE2EDuration="7.323478776s" totalImagesPullingTime="2.451023549s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:12:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:12:30.320881694 +0000 UTC m=+74.734750856" watchObservedRunningTime="2026-07-07 11:12:30.323478776 +0000 UTC m=+74.737347942" Jul 07 11:12:31.732297 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:31.732242 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a4b1e5f7-71c6-4349-8dad-ef5763f9bfb9-original-pull-secret\") pod \"global-pull-secret-syncer-7ml6d\" (UID: \"a4b1e5f7-71c6-4349-8dad-ef5763f9bfb9\") " pod="kube-system/global-pull-secret-syncer-7ml6d" Jul 07 11:12:31.735140 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:31.735116 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 07 11:12:31.745375 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:31.745320 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a4b1e5f7-71c6-4349-8dad-ef5763f9bfb9-original-pull-secret\") pod \"global-pull-secret-syncer-7ml6d\" (UID: \"a4b1e5f7-71c6-4349-8dad-ef5763f9bfb9\") " pod="kube-system/global-pull-secret-syncer-7ml6d" Jul 07 11:12:31.772119 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:31.772008 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7ml6d" Jul 07 11:12:32.306783 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:32.306752 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-7ml6d"] Jul 07 11:12:32.310673 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:32.307689 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c088023f-86df-4531-be68-1a21bad47ae6","Type":"ContainerStarted","Data":"e98438c43873adcbb713a7890ec5d37b1f1619423ae36cc69c9dc54490253c68"} Jul 07 11:12:32.311982 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:12:32.311949 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4b1e5f7_71c6_4349_8dad_ef5763f9bfb9.slice/crio-b91c94cd1f066b8babaaddcf00ddb80233dc785cbf545774fa363502854ac01d WatchSource:0}: Error finding container b91c94cd1f066b8babaaddcf00ddb80233dc785cbf545774fa363502854ac01d: Status 404 returned error can't find the container with id b91c94cd1f066b8babaaddcf00ddb80233dc785cbf545774fa363502854ac01d Jul 07 11:12:32.622029 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:32.622000 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:12:32.799787 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:32.799732 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:12:33.317906 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:33.317604 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c088023f-86df-4531-be68-1a21bad47ae6","Type":"ContainerStarted","Data":"39d4460ed91020e9f75de9cec8cd0334ef6c845ef86544427f906b3b7d1eb463"} Jul 07 11:12:33.317906 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:33.317718 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c088023f-86df-4531-be68-1a21bad47ae6","Type":"ContainerStarted","Data":"06bf9b5a2354ff69ec76d04926974872d40a7e36cedbb69c51792d0c84383954"} Jul 07 11:12:33.317906 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:33.317762 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c088023f-86df-4531-be68-1a21bad47ae6","Type":"ContainerStarted","Data":"9bddc02f54e220f34e3c00d4af2883a1576cd50ea4fcba3ed0450e695db4105d"} Jul 07 11:12:33.317906 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:33.317798 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c088023f-86df-4531-be68-1a21bad47ae6","Type":"ContainerStarted","Data":"f7f965b05c081819965360a985cc73476b8f1b08b69eeff76dfd2d119f816822"} Jul 07 11:12:33.317906 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:33.317856 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c088023f-86df-4531-be68-1a21bad47ae6","Type":"ContainerStarted","Data":"69ecb11f604d74c323cefb8de01623ef2a58f38b7b2ba3a9214143b2b494ec6d"} Jul 07 11:12:33.319155 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:33.319126 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-7ml6d" event={"ID":"a4b1e5f7-71c6-4349-8dad-ef5763f9bfb9","Type":"ContainerStarted","Data":"b91c94cd1f066b8babaaddcf00ddb80233dc785cbf545774fa363502854ac01d"} Jul 07 11:12:33.813642 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:33.813604 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:12:33.931882 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:33.931853 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:12:34.022152 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:34.022105 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:12:35.154315 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:35.154254 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:12:35.251060 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:35.251020 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:12:35.427557 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:35.427476 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:12:36.306894 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:36.306866 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-746f69848d-65tjq" Jul 07 11:12:36.344709 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:36.344649 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=7.4043638 podStartE2EDuration="11.344625029s" totalImagesPullingTime="3.915226532s" totalInitContainerRuntime="25.034697ms" isStatefulPod=false podCreationTimestamp="2026-07-07 11:12:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:12:33.356663464 +0000 UTC m=+77.770532630" watchObservedRunningTime="2026-07-07 11:12:36.344625029 +0000 UTC m=+80.758494194" Jul 07 11:12:37.343559 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:37.343428 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-7ml6d" event={"ID":"a4b1e5f7-71c6-4349-8dad-ef5763f9bfb9","Type":"ContainerStarted","Data":"10f89e9f344798cd1b3af17ffccde60fb4ee0dee04b43cebcb3f5753228e6edc"} Jul 07 11:12:37.368133 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:37.368077 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-7ml6d" podStartSLOduration=65.631581827 podStartE2EDuration="1m10.368055521s" totalImagesPullingTime="4.736473694s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 11:11:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:12:37.366124167 +0000 UTC m=+81.779993334" watchObservedRunningTime="2026-07-07 11:12:37.368055521 +0000 UTC m=+81.781924692" Jul 07 11:12:39.261954 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:39.261806 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:12:39.329737 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:39.329695 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:12:40.301141 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:40.301108 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:12:40.403631 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:40.403587 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:12:42.566580 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:42.566535 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:12:42.662256 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:42.662226 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:12:43.400723 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:43.400690 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-8654d65bc6-c676s" Jul 07 11:12:43.401078 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:43.400788 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-8654d65bc6-c676s" Jul 07 11:12:44.487307 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:44.487275 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:12:44.541187 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:44.541158 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:12:49.401332 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:49.401288 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-685bfb5f64-428ms" event={"ID":"a40bc7e6-49a7-49f7-b204-e03679e7f1fc","Type":"ContainerStarted","Data":"43539889db1f537f2e77941fc2777e7e23f8367c8c5b42e4dcf04e4682384723"} Jul 07 11:12:49.401858 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:49.401528 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-685bfb5f64-428ms" Jul 07 11:12:49.415657 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:49.415628 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-685bfb5f64-428ms" Jul 07 11:12:49.420407 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:49.420351 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-685bfb5f64-428ms" podStartSLOduration=4.521517491 podStartE2EDuration="25.420335781s" totalImagesPullingTime="20.89881829s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:12:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:12:49.418685202 +0000 UTC m=+93.832554381" watchObservedRunningTime="2026-07-07 11:12:49.420335781 +0000 UTC m=+93.834204945" Jul 07 11:12:51.307952 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:51.307877 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:12:51.387655 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:51.387616 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:12:55.300097 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:55.300025 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:12:56.826178 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:56.826141 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:12:56.927126 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:56.927084 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:12:57.055714 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:57.055675 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:12:57.610173 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:57.610140 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:12:57.731845 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:57.731812 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:12:59.286050 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:12:59.286021 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-sc5bl" Jul 07 11:13:03.405638 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:03.405607 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-8654d65bc6-c676s" Jul 07 11:13:03.409485 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:03.409465 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-8654d65bc6-c676s" Jul 07 11:13:12.615813 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:12.615765 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:13:12.858046 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:12.858009 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:13:13.436715 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:13.436684 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:13:25.427455 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:25.427422 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:13:25.489760 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:25.489736 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:13:25.566323 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:25.566296 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:13:34.990282 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:34.990246 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:13:35.109049 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:35.109017 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:13:39.300619 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:39.300582 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 07 11:13:39.301030 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:39.300899 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="9314229f-a3c5-4809-8129-b06de0b52127" containerName="alertmanager" containerID="cri-o://1852311dd0b3159a0c90baf81ae46cf25809a80ec728f190fc163a7dfae2a336" gracePeriod=120 Jul 07 11:13:39.301108 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:39.300979 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="9314229f-a3c5-4809-8129-b06de0b52127" containerName="kube-rbac-proxy-metric" containerID="cri-o://fda1eb6c9a21a1a11d784870ce053940d72f5040e7cfe866003bb2a64b4a8ed3" gracePeriod=120 Jul 07 11:13:39.301108 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:39.301033 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="9314229f-a3c5-4809-8129-b06de0b52127" containerName="kube-rbac-proxy-web" containerID="cri-o://6ecec6f2bf77b54f5f6df144e16b23b8ddd9fa14e01930a3ee23841f970a5fc2" gracePeriod=120 Jul 07 11:13:39.301108 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:39.301049 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="9314229f-a3c5-4809-8129-b06de0b52127" containerName="config-reloader" containerID="cri-o://0dbb621e5297345bd37afa912a1dcdeb365469f041479464d014b99beb6245af" gracePeriod=120 Jul 07 11:13:39.301108 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:39.301084 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="9314229f-a3c5-4809-8129-b06de0b52127" containerName="kube-rbac-proxy" containerID="cri-o://a837483aac629f55a9de413741ae4ac7ac5c5b47f1015ee044a7a2845388f98f" gracePeriod=120 Jul 07 11:13:39.301292 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:39.301035 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="9314229f-a3c5-4809-8129-b06de0b52127" containerName="prom-label-proxy" containerID="cri-o://49442c22eefabdd95e299db5a21b5eae8a7f2cbb6806da0aa00fba206790df3f" gracePeriod=120 Jul 07 11:13:39.617690 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:39.617619 2562 generic.go:356] "Generic (PLEG): container finished" podID="9314229f-a3c5-4809-8129-b06de0b52127" containerID="49442c22eefabdd95e299db5a21b5eae8a7f2cbb6806da0aa00fba206790df3f" exitCode=0 Jul 07 11:13:39.617690 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:39.617646 2562 generic.go:356] "Generic (PLEG): container finished" podID="9314229f-a3c5-4809-8129-b06de0b52127" containerID="a837483aac629f55a9de413741ae4ac7ac5c5b47f1015ee044a7a2845388f98f" exitCode=0 Jul 07 11:13:39.617690 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:39.617656 2562 generic.go:356] "Generic (PLEG): container finished" podID="9314229f-a3c5-4809-8129-b06de0b52127" containerID="0dbb621e5297345bd37afa912a1dcdeb365469f041479464d014b99beb6245af" exitCode=0 Jul 07 11:13:39.617690 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:39.617663 2562 generic.go:356] "Generic (PLEG): container finished" podID="9314229f-a3c5-4809-8129-b06de0b52127" containerID="1852311dd0b3159a0c90baf81ae46cf25809a80ec728f190fc163a7dfae2a336" exitCode=0 Jul 07 11:13:39.617903 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:39.617689 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9314229f-a3c5-4809-8129-b06de0b52127","Type":"ContainerDied","Data":"49442c22eefabdd95e299db5a21b5eae8a7f2cbb6806da0aa00fba206790df3f"} Jul 07 11:13:39.617903 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:39.617754 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9314229f-a3c5-4809-8129-b06de0b52127","Type":"ContainerDied","Data":"a837483aac629f55a9de413741ae4ac7ac5c5b47f1015ee044a7a2845388f98f"} Jul 07 11:13:39.617903 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:39.617781 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9314229f-a3c5-4809-8129-b06de0b52127","Type":"ContainerDied","Data":"0dbb621e5297345bd37afa912a1dcdeb365469f041479464d014b99beb6245af"} Jul 07 11:13:39.617903 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:39.617807 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9314229f-a3c5-4809-8129-b06de0b52127","Type":"ContainerDied","Data":"1852311dd0b3159a0c90baf81ae46cf25809a80ec728f190fc163a7dfae2a336"} Jul 07 11:13:40.542309 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:40.542288 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:13:40.625290 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:40.625210 2562 generic.go:356] "Generic (PLEG): container finished" podID="9314229f-a3c5-4809-8129-b06de0b52127" containerID="fda1eb6c9a21a1a11d784870ce053940d72f5040e7cfe866003bb2a64b4a8ed3" exitCode=0 Jul 07 11:13:40.625290 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:40.625232 2562 generic.go:356] "Generic (PLEG): container finished" podID="9314229f-a3c5-4809-8129-b06de0b52127" containerID="6ecec6f2bf77b54f5f6df144e16b23b8ddd9fa14e01930a3ee23841f970a5fc2" exitCode=0 Jul 07 11:13:40.625490 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:40.625304 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:13:40.625490 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:40.625300 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9314229f-a3c5-4809-8129-b06de0b52127","Type":"ContainerDied","Data":"fda1eb6c9a21a1a11d784870ce053940d72f5040e7cfe866003bb2a64b4a8ed3"} Jul 07 11:13:40.625490 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:40.625399 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9314229f-a3c5-4809-8129-b06de0b52127","Type":"ContainerDied","Data":"6ecec6f2bf77b54f5f6df144e16b23b8ddd9fa14e01930a3ee23841f970a5fc2"} Jul 07 11:13:40.625490 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:40.625443 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9314229f-a3c5-4809-8129-b06de0b52127","Type":"ContainerDied","Data":"8ba017bb8889f7c2be39bf401be69f96d4b9e92a2af815a30151bbb36e1fe4f1"} Jul 07 11:13:40.625490 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:40.625472 2562 scope.go:117] "RemoveContainer" containerID="49442c22eefabdd95e299db5a21b5eae8a7f2cbb6806da0aa00fba206790df3f" Jul 07 11:13:40.633367 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:40.633350 2562 scope.go:117] "RemoveContainer" containerID="fda1eb6c9a21a1a11d784870ce053940d72f5040e7cfe866003bb2a64b4a8ed3" Jul 07 11:13:40.640027 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:40.640007 2562 scope.go:117] "RemoveContainer" containerID="a837483aac629f55a9de413741ae4ac7ac5c5b47f1015ee044a7a2845388f98f" Jul 07 11:13:40.640173 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:40.640156 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dtqff\" (UniqueName: \"kubernetes.io/projected/9314229f-a3c5-4809-8129-b06de0b52127-kube-api-access-dtqff\") pod \"9314229f-a3c5-4809-8129-b06de0b52127\" (UID: \"9314229f-a3c5-4809-8129-b06de0b52127\") " Jul 07 11:13:40.640222 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:40.640185 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9314229f-a3c5-4809-8129-b06de0b52127-tls-assets\") pod \"9314229f-a3c5-4809-8129-b06de0b52127\" (UID: \"9314229f-a3c5-4809-8129-b06de0b52127\") " Jul 07 11:13:40.640222 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:40.640215 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9314229f-a3c5-4809-8129-b06de0b52127-alertmanager-trusted-ca-bundle\") pod \"9314229f-a3c5-4809-8129-b06de0b52127\" (UID: \"9314229f-a3c5-4809-8129-b06de0b52127\") " Jul 07 11:13:40.640317 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:40.640243 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9314229f-a3c5-4809-8129-b06de0b52127-config-out\") pod \"9314229f-a3c5-4809-8129-b06de0b52127\" (UID: \"9314229f-a3c5-4809-8129-b06de0b52127\") " Jul 07 11:13:40.640317 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:40.640271 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9314229f-a3c5-4809-8129-b06de0b52127-metrics-client-ca\") pod \"9314229f-a3c5-4809-8129-b06de0b52127\" (UID: \"9314229f-a3c5-4809-8129-b06de0b52127\") " Jul 07 11:13:40.640317 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:40.640306 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/9314229f-a3c5-4809-8129-b06de0b52127-cluster-tls-config\") pod \"9314229f-a3c5-4809-8129-b06de0b52127\" (UID: \"9314229f-a3c5-4809-8129-b06de0b52127\") " Jul 07 11:13:40.640470 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:40.640342 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/9314229f-a3c5-4809-8129-b06de0b52127-secret-alertmanager-kube-rbac-proxy-metric\") pod \"9314229f-a3c5-4809-8129-b06de0b52127\" (UID: \"9314229f-a3c5-4809-8129-b06de0b52127\") " Jul 07 11:13:40.640470 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:40.640371 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9314229f-a3c5-4809-8129-b06de0b52127-web-config\") pod \"9314229f-a3c5-4809-8129-b06de0b52127\" (UID: \"9314229f-a3c5-4809-8129-b06de0b52127\") " Jul 07 11:13:40.640470 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:40.640400 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/9314229f-a3c5-4809-8129-b06de0b52127-alertmanager-main-db\") pod \"9314229f-a3c5-4809-8129-b06de0b52127\" (UID: \"9314229f-a3c5-4809-8129-b06de0b52127\") " Jul 07 11:13:40.640470 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:40.640443 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9314229f-a3c5-4809-8129-b06de0b52127-secret-alertmanager-kube-rbac-proxy-web\") pod \"9314229f-a3c5-4809-8129-b06de0b52127\" (UID: \"9314229f-a3c5-4809-8129-b06de0b52127\") " Jul 07 11:13:40.640470 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:40.640469 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/9314229f-a3c5-4809-8129-b06de0b52127-config-volume\") pod \"9314229f-a3c5-4809-8129-b06de0b52127\" (UID: \"9314229f-a3c5-4809-8129-b06de0b52127\") " Jul 07 11:13:40.640747 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:40.640519 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/9314229f-a3c5-4809-8129-b06de0b52127-secret-alertmanager-main-tls\") pod \"9314229f-a3c5-4809-8129-b06de0b52127\" (UID: \"9314229f-a3c5-4809-8129-b06de0b52127\") " Jul 07 11:13:40.640747 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:40.640567 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9314229f-a3c5-4809-8129-b06de0b52127-secret-alertmanager-kube-rbac-proxy\") pod \"9314229f-a3c5-4809-8129-b06de0b52127\" (UID: \"9314229f-a3c5-4809-8129-b06de0b52127\") " Jul 07 11:13:40.640747 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:40.640685 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9314229f-a3c5-4809-8129-b06de0b52127-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "9314229f-a3c5-4809-8129-b06de0b52127" (UID: "9314229f-a3c5-4809-8129-b06de0b52127"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 11:13:40.640935 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:40.640872 2562 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9314229f-a3c5-4809-8129-b06de0b52127-metrics-client-ca\") on node \"ip-10-0-136-15.ec2.internal\" DevicePath \"\"" Jul 07 11:13:40.641313 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:40.641042 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9314229f-a3c5-4809-8129-b06de0b52127-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "9314229f-a3c5-4809-8129-b06de0b52127" (UID: "9314229f-a3c5-4809-8129-b06de0b52127"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 11:13:40.641733 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:40.641708 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9314229f-a3c5-4809-8129-b06de0b52127-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "9314229f-a3c5-4809-8129-b06de0b52127" (UID: "9314229f-a3c5-4809-8129-b06de0b52127"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 11:13:40.643708 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:40.643666 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9314229f-a3c5-4809-8129-b06de0b52127-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "9314229f-a3c5-4809-8129-b06de0b52127" (UID: "9314229f-a3c5-4809-8129-b06de0b52127"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 11:13:40.643708 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:40.643678 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9314229f-a3c5-4809-8129-b06de0b52127-config-out" (OuterVolumeSpecName: "config-out") pod "9314229f-a3c5-4809-8129-b06de0b52127" (UID: "9314229f-a3c5-4809-8129-b06de0b52127"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 11:13:40.644144 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:40.644074 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9314229f-a3c5-4809-8129-b06de0b52127-kube-api-access-dtqff" (OuterVolumeSpecName: "kube-api-access-dtqff") pod "9314229f-a3c5-4809-8129-b06de0b52127" (UID: "9314229f-a3c5-4809-8129-b06de0b52127"). InnerVolumeSpecName "kube-api-access-dtqff". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 11:13:40.644144 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:40.644077 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9314229f-a3c5-4809-8129-b06de0b52127-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "9314229f-a3c5-4809-8129-b06de0b52127" (UID: "9314229f-a3c5-4809-8129-b06de0b52127"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 11:13:40.644461 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:40.644438 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9314229f-a3c5-4809-8129-b06de0b52127-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "9314229f-a3c5-4809-8129-b06de0b52127" (UID: "9314229f-a3c5-4809-8129-b06de0b52127"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 11:13:40.644773 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:40.644743 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9314229f-a3c5-4809-8129-b06de0b52127-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "9314229f-a3c5-4809-8129-b06de0b52127" (UID: "9314229f-a3c5-4809-8129-b06de0b52127"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 11:13:40.645112 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:40.645079 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9314229f-a3c5-4809-8129-b06de0b52127-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "9314229f-a3c5-4809-8129-b06de0b52127" (UID: "9314229f-a3c5-4809-8129-b06de0b52127"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 11:13:40.645201 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:40.645156 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9314229f-a3c5-4809-8129-b06de0b52127-config-volume" (OuterVolumeSpecName: "config-volume") pod "9314229f-a3c5-4809-8129-b06de0b52127" (UID: "9314229f-a3c5-4809-8129-b06de0b52127"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 11:13:40.648735 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:40.648715 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9314229f-a3c5-4809-8129-b06de0b52127-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "9314229f-a3c5-4809-8129-b06de0b52127" (UID: "9314229f-a3c5-4809-8129-b06de0b52127"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 11:13:40.655839 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:40.655821 2562 scope.go:117] "RemoveContainer" containerID="6ecec6f2bf77b54f5f6df144e16b23b8ddd9fa14e01930a3ee23841f970a5fc2" Jul 07 11:13:40.656386 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:40.656367 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9314229f-a3c5-4809-8129-b06de0b52127-web-config" (OuterVolumeSpecName: "web-config") pod "9314229f-a3c5-4809-8129-b06de0b52127" (UID: "9314229f-a3c5-4809-8129-b06de0b52127"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 11:13:40.662151 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:40.662132 2562 scope.go:117] "RemoveContainer" containerID="0dbb621e5297345bd37afa912a1dcdeb365469f041479464d014b99beb6245af" Jul 07 11:13:40.668621 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:40.668607 2562 scope.go:117] "RemoveContainer" containerID="1852311dd0b3159a0c90baf81ae46cf25809a80ec728f190fc163a7dfae2a336" Jul 07 11:13:40.675062 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:40.675026 2562 scope.go:117] "RemoveContainer" containerID="73d2397143dec700148b876a6451b89628026aae7d4e6f5cb0812bfcf65eb0ce" Jul 07 11:13:40.681426 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:40.681410 2562 scope.go:117] "RemoveContainer" containerID="49442c22eefabdd95e299db5a21b5eae8a7f2cbb6806da0aa00fba206790df3f" Jul 07 11:13:40.681686 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:13:40.681669 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"49442c22eefabdd95e299db5a21b5eae8a7f2cbb6806da0aa00fba206790df3f\": container with ID starting with 49442c22eefabdd95e299db5a21b5eae8a7f2cbb6806da0aa00fba206790df3f not found: ID does not exist" containerID="49442c22eefabdd95e299db5a21b5eae8a7f2cbb6806da0aa00fba206790df3f" Jul 07 11:13:40.681725 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:40.681694 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"49442c22eefabdd95e299db5a21b5eae8a7f2cbb6806da0aa00fba206790df3f"} err="failed to get container status \"49442c22eefabdd95e299db5a21b5eae8a7f2cbb6806da0aa00fba206790df3f\": rpc error: code = NotFound desc = could not find container \"49442c22eefabdd95e299db5a21b5eae8a7f2cbb6806da0aa00fba206790df3f\": container with ID starting with 49442c22eefabdd95e299db5a21b5eae8a7f2cbb6806da0aa00fba206790df3f not found: ID does not exist" Jul 07 11:13:40.681765 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:40.681729 2562 scope.go:117] "RemoveContainer" containerID="fda1eb6c9a21a1a11d784870ce053940d72f5040e7cfe866003bb2a64b4a8ed3" Jul 07 11:13:40.681976 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:13:40.681959 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fda1eb6c9a21a1a11d784870ce053940d72f5040e7cfe866003bb2a64b4a8ed3\": container with ID starting with fda1eb6c9a21a1a11d784870ce053940d72f5040e7cfe866003bb2a64b4a8ed3 not found: ID does not exist" containerID="fda1eb6c9a21a1a11d784870ce053940d72f5040e7cfe866003bb2a64b4a8ed3" Jul 07 11:13:40.682035 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:40.681982 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fda1eb6c9a21a1a11d784870ce053940d72f5040e7cfe866003bb2a64b4a8ed3"} err="failed to get container status \"fda1eb6c9a21a1a11d784870ce053940d72f5040e7cfe866003bb2a64b4a8ed3\": rpc error: code = NotFound desc = could not find container \"fda1eb6c9a21a1a11d784870ce053940d72f5040e7cfe866003bb2a64b4a8ed3\": container with ID starting with fda1eb6c9a21a1a11d784870ce053940d72f5040e7cfe866003bb2a64b4a8ed3 not found: ID does not exist" Jul 07 11:13:40.682035 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:40.681999 2562 scope.go:117] "RemoveContainer" containerID="a837483aac629f55a9de413741ae4ac7ac5c5b47f1015ee044a7a2845388f98f" Jul 07 11:13:40.682237 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:13:40.682220 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a837483aac629f55a9de413741ae4ac7ac5c5b47f1015ee044a7a2845388f98f\": container with ID starting with a837483aac629f55a9de413741ae4ac7ac5c5b47f1015ee044a7a2845388f98f not found: ID does not exist" containerID="a837483aac629f55a9de413741ae4ac7ac5c5b47f1015ee044a7a2845388f98f" Jul 07 11:13:40.682286 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:40.682242 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a837483aac629f55a9de413741ae4ac7ac5c5b47f1015ee044a7a2845388f98f"} err="failed to get container status \"a837483aac629f55a9de413741ae4ac7ac5c5b47f1015ee044a7a2845388f98f\": rpc error: code = NotFound desc = could not find container \"a837483aac629f55a9de413741ae4ac7ac5c5b47f1015ee044a7a2845388f98f\": container with ID starting with a837483aac629f55a9de413741ae4ac7ac5c5b47f1015ee044a7a2845388f98f not found: ID does not exist" Jul 07 11:13:40.682286 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:40.682257 2562 scope.go:117] "RemoveContainer" containerID="6ecec6f2bf77b54f5f6df144e16b23b8ddd9fa14e01930a3ee23841f970a5fc2" Jul 07 11:13:40.682489 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:13:40.682473 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6ecec6f2bf77b54f5f6df144e16b23b8ddd9fa14e01930a3ee23841f970a5fc2\": container with ID starting with 6ecec6f2bf77b54f5f6df144e16b23b8ddd9fa14e01930a3ee23841f970a5fc2 not found: ID does not exist" containerID="6ecec6f2bf77b54f5f6df144e16b23b8ddd9fa14e01930a3ee23841f970a5fc2" Jul 07 11:13:40.682528 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:40.682491 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6ecec6f2bf77b54f5f6df144e16b23b8ddd9fa14e01930a3ee23841f970a5fc2"} err="failed to get container status \"6ecec6f2bf77b54f5f6df144e16b23b8ddd9fa14e01930a3ee23841f970a5fc2\": rpc error: code = NotFound desc = could not find container \"6ecec6f2bf77b54f5f6df144e16b23b8ddd9fa14e01930a3ee23841f970a5fc2\": container with ID starting with 6ecec6f2bf77b54f5f6df144e16b23b8ddd9fa14e01930a3ee23841f970a5fc2 not found: ID does not exist" Jul 07 11:13:40.682528 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:40.682503 2562 scope.go:117] "RemoveContainer" containerID="0dbb621e5297345bd37afa912a1dcdeb365469f041479464d014b99beb6245af" Jul 07 11:13:40.682727 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:13:40.682710 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0dbb621e5297345bd37afa912a1dcdeb365469f041479464d014b99beb6245af\": container with ID starting with 0dbb621e5297345bd37afa912a1dcdeb365469f041479464d014b99beb6245af not found: ID does not exist" containerID="0dbb621e5297345bd37afa912a1dcdeb365469f041479464d014b99beb6245af" Jul 07 11:13:40.682767 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:40.682732 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0dbb621e5297345bd37afa912a1dcdeb365469f041479464d014b99beb6245af"} err="failed to get container status \"0dbb621e5297345bd37afa912a1dcdeb365469f041479464d014b99beb6245af\": rpc error: code = NotFound desc = could not find container \"0dbb621e5297345bd37afa912a1dcdeb365469f041479464d014b99beb6245af\": container with ID starting with 0dbb621e5297345bd37afa912a1dcdeb365469f041479464d014b99beb6245af not found: ID does not exist" Jul 07 11:13:40.682767 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:40.682754 2562 scope.go:117] "RemoveContainer" containerID="1852311dd0b3159a0c90baf81ae46cf25809a80ec728f190fc163a7dfae2a336" Jul 07 11:13:40.682993 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:13:40.682976 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1852311dd0b3159a0c90baf81ae46cf25809a80ec728f190fc163a7dfae2a336\": container with ID starting with 1852311dd0b3159a0c90baf81ae46cf25809a80ec728f190fc163a7dfae2a336 not found: ID does not exist" containerID="1852311dd0b3159a0c90baf81ae46cf25809a80ec728f190fc163a7dfae2a336" Jul 07 11:13:40.683060 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:40.683000 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1852311dd0b3159a0c90baf81ae46cf25809a80ec728f190fc163a7dfae2a336"} err="failed to get container status \"1852311dd0b3159a0c90baf81ae46cf25809a80ec728f190fc163a7dfae2a336\": rpc error: code = NotFound desc = could not find container \"1852311dd0b3159a0c90baf81ae46cf25809a80ec728f190fc163a7dfae2a336\": container with ID starting with 1852311dd0b3159a0c90baf81ae46cf25809a80ec728f190fc163a7dfae2a336 not found: ID does not exist" Jul 07 11:13:40.683060 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:40.683021 2562 scope.go:117] "RemoveContainer" containerID="73d2397143dec700148b876a6451b89628026aae7d4e6f5cb0812bfcf65eb0ce" Jul 07 11:13:40.683252 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:13:40.683233 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"73d2397143dec700148b876a6451b89628026aae7d4e6f5cb0812bfcf65eb0ce\": container with ID starting with 73d2397143dec700148b876a6451b89628026aae7d4e6f5cb0812bfcf65eb0ce not found: ID does not exist" containerID="73d2397143dec700148b876a6451b89628026aae7d4e6f5cb0812bfcf65eb0ce" Jul 07 11:13:40.683293 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:40.683256 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"73d2397143dec700148b876a6451b89628026aae7d4e6f5cb0812bfcf65eb0ce"} err="failed to get container status \"73d2397143dec700148b876a6451b89628026aae7d4e6f5cb0812bfcf65eb0ce\": rpc error: code = NotFound desc = could not find container \"73d2397143dec700148b876a6451b89628026aae7d4e6f5cb0812bfcf65eb0ce\": container with ID starting with 73d2397143dec700148b876a6451b89628026aae7d4e6f5cb0812bfcf65eb0ce not found: ID does not exist" Jul 07 11:13:40.683293 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:40.683268 2562 scope.go:117] "RemoveContainer" containerID="49442c22eefabdd95e299db5a21b5eae8a7f2cbb6806da0aa00fba206790df3f" Jul 07 11:13:40.683480 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:40.683462 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"49442c22eefabdd95e299db5a21b5eae8a7f2cbb6806da0aa00fba206790df3f"} err="failed to get container status \"49442c22eefabdd95e299db5a21b5eae8a7f2cbb6806da0aa00fba206790df3f\": rpc error: code = NotFound desc = could not find container \"49442c22eefabdd95e299db5a21b5eae8a7f2cbb6806da0aa00fba206790df3f\": container with ID starting with 49442c22eefabdd95e299db5a21b5eae8a7f2cbb6806da0aa00fba206790df3f not found: ID does not exist" Jul 07 11:13:40.683546 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:40.683482 2562 scope.go:117] "RemoveContainer" containerID="fda1eb6c9a21a1a11d784870ce053940d72f5040e7cfe866003bb2a64b4a8ed3" Jul 07 11:13:40.683662 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:40.683646 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fda1eb6c9a21a1a11d784870ce053940d72f5040e7cfe866003bb2a64b4a8ed3"} err="failed to get container status \"fda1eb6c9a21a1a11d784870ce053940d72f5040e7cfe866003bb2a64b4a8ed3\": rpc error: code = NotFound desc = could not find container \"fda1eb6c9a21a1a11d784870ce053940d72f5040e7cfe866003bb2a64b4a8ed3\": container with ID starting with fda1eb6c9a21a1a11d784870ce053940d72f5040e7cfe866003bb2a64b4a8ed3 not found: ID does not exist" Jul 07 11:13:40.683707 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:40.683662 2562 scope.go:117] "RemoveContainer" containerID="a837483aac629f55a9de413741ae4ac7ac5c5b47f1015ee044a7a2845388f98f" Jul 07 11:13:40.683840 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:40.683824 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a837483aac629f55a9de413741ae4ac7ac5c5b47f1015ee044a7a2845388f98f"} err="failed to get container status \"a837483aac629f55a9de413741ae4ac7ac5c5b47f1015ee044a7a2845388f98f\": rpc error: code = NotFound desc = could not find container \"a837483aac629f55a9de413741ae4ac7ac5c5b47f1015ee044a7a2845388f98f\": container with ID starting with a837483aac629f55a9de413741ae4ac7ac5c5b47f1015ee044a7a2845388f98f not found: ID does not exist" Jul 07 11:13:40.683898 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:40.683841 2562 scope.go:117] "RemoveContainer" containerID="6ecec6f2bf77b54f5f6df144e16b23b8ddd9fa14e01930a3ee23841f970a5fc2" Jul 07 11:13:40.684047 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:40.684030 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6ecec6f2bf77b54f5f6df144e16b23b8ddd9fa14e01930a3ee23841f970a5fc2"} err="failed to get container status \"6ecec6f2bf77b54f5f6df144e16b23b8ddd9fa14e01930a3ee23841f970a5fc2\": rpc error: code = NotFound desc = could not find container \"6ecec6f2bf77b54f5f6df144e16b23b8ddd9fa14e01930a3ee23841f970a5fc2\": container with ID starting with 6ecec6f2bf77b54f5f6df144e16b23b8ddd9fa14e01930a3ee23841f970a5fc2 not found: ID does not exist" Jul 07 11:13:40.684118 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:40.684049 2562 scope.go:117] "RemoveContainer" containerID="0dbb621e5297345bd37afa912a1dcdeb365469f041479464d014b99beb6245af" Jul 07 11:13:40.684271 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:40.684254 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0dbb621e5297345bd37afa912a1dcdeb365469f041479464d014b99beb6245af"} err="failed to get container status \"0dbb621e5297345bd37afa912a1dcdeb365469f041479464d014b99beb6245af\": rpc error: code = NotFound desc = could not find container \"0dbb621e5297345bd37afa912a1dcdeb365469f041479464d014b99beb6245af\": container with ID starting with 0dbb621e5297345bd37afa912a1dcdeb365469f041479464d014b99beb6245af not found: ID does not exist" Jul 07 11:13:40.684336 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:40.684272 2562 scope.go:117] "RemoveContainer" containerID="1852311dd0b3159a0c90baf81ae46cf25809a80ec728f190fc163a7dfae2a336" Jul 07 11:13:40.684478 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:40.684459 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1852311dd0b3159a0c90baf81ae46cf25809a80ec728f190fc163a7dfae2a336"} err="failed to get container status \"1852311dd0b3159a0c90baf81ae46cf25809a80ec728f190fc163a7dfae2a336\": rpc error: code = NotFound desc = could not find container \"1852311dd0b3159a0c90baf81ae46cf25809a80ec728f190fc163a7dfae2a336\": container with ID starting with 1852311dd0b3159a0c90baf81ae46cf25809a80ec728f190fc163a7dfae2a336 not found: ID does not exist" Jul 07 11:13:40.684521 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:40.684479 2562 scope.go:117] "RemoveContainer" containerID="73d2397143dec700148b876a6451b89628026aae7d4e6f5cb0812bfcf65eb0ce" Jul 07 11:13:40.684663 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:40.684644 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"73d2397143dec700148b876a6451b89628026aae7d4e6f5cb0812bfcf65eb0ce"} err="failed to get container status \"73d2397143dec700148b876a6451b89628026aae7d4e6f5cb0812bfcf65eb0ce\": rpc error: code = NotFound desc = could not find container \"73d2397143dec700148b876a6451b89628026aae7d4e6f5cb0812bfcf65eb0ce\": container with ID starting with 73d2397143dec700148b876a6451b89628026aae7d4e6f5cb0812bfcf65eb0ce not found: ID does not exist" Jul 07 11:13:40.741796 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:40.741773 2562 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/9314229f-a3c5-4809-8129-b06de0b52127-cluster-tls-config\") on node \"ip-10-0-136-15.ec2.internal\" DevicePath \"\"" Jul 07 11:13:40.741796 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:40.741794 2562 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/9314229f-a3c5-4809-8129-b06de0b52127-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-136-15.ec2.internal\" DevicePath \"\"" Jul 07 11:13:40.741899 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:40.741804 2562 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9314229f-a3c5-4809-8129-b06de0b52127-web-config\") on node \"ip-10-0-136-15.ec2.internal\" DevicePath \"\"" Jul 07 11:13:40.741899 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:40.741813 2562 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/9314229f-a3c5-4809-8129-b06de0b52127-alertmanager-main-db\") on node \"ip-10-0-136-15.ec2.internal\" DevicePath \"\"" Jul 07 11:13:40.741899 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:40.741823 2562 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9314229f-a3c5-4809-8129-b06de0b52127-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-136-15.ec2.internal\" DevicePath \"\"" Jul 07 11:13:40.741899 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:40.741834 2562 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/9314229f-a3c5-4809-8129-b06de0b52127-config-volume\") on node \"ip-10-0-136-15.ec2.internal\" DevicePath \"\"" Jul 07 11:13:40.741899 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:40.741842 2562 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/9314229f-a3c5-4809-8129-b06de0b52127-secret-alertmanager-main-tls\") on node \"ip-10-0-136-15.ec2.internal\" DevicePath \"\"" Jul 07 11:13:40.741899 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:40.741850 2562 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9314229f-a3c5-4809-8129-b06de0b52127-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-136-15.ec2.internal\" DevicePath \"\"" Jul 07 11:13:40.741899 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:40.741858 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dtqff\" (UniqueName: \"kubernetes.io/projected/9314229f-a3c5-4809-8129-b06de0b52127-kube-api-access-dtqff\") on node \"ip-10-0-136-15.ec2.internal\" DevicePath \"\"" Jul 07 11:13:40.741899 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:40.741866 2562 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9314229f-a3c5-4809-8129-b06de0b52127-tls-assets\") on node \"ip-10-0-136-15.ec2.internal\" DevicePath \"\"" Jul 07 11:13:40.741899 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:40.741874 2562 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9314229f-a3c5-4809-8129-b06de0b52127-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-136-15.ec2.internal\" DevicePath \"\"" Jul 07 11:13:40.741899 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:40.741882 2562 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9314229f-a3c5-4809-8129-b06de0b52127-config-out\") on node \"ip-10-0-136-15.ec2.internal\" DevicePath \"\"" Jul 07 11:13:40.949515 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:40.949492 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 07 11:13:40.954458 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:40.954439 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 07 11:13:40.981100 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:40.981074 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 07 11:13:40.981365 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:40.981353 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9314229f-a3c5-4809-8129-b06de0b52127" containerName="kube-rbac-proxy-web" Jul 07 11:13:40.981409 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:40.981366 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="9314229f-a3c5-4809-8129-b06de0b52127" containerName="kube-rbac-proxy-web" Jul 07 11:13:40.981409 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:40.981379 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9314229f-a3c5-4809-8129-b06de0b52127" containerName="init-config-reloader" Jul 07 11:13:40.981409 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:40.981385 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="9314229f-a3c5-4809-8129-b06de0b52127" containerName="init-config-reloader" Jul 07 11:13:40.981409 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:40.981392 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9314229f-a3c5-4809-8129-b06de0b52127" containerName="alertmanager" Jul 07 11:13:40.981409 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:40.981397 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="9314229f-a3c5-4809-8129-b06de0b52127" containerName="alertmanager" Jul 07 11:13:40.981409 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:40.981403 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9314229f-a3c5-4809-8129-b06de0b52127" containerName="kube-rbac-proxy-metric" Jul 07 11:13:40.981409 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:40.981408 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="9314229f-a3c5-4809-8129-b06de0b52127" containerName="kube-rbac-proxy-metric" Jul 07 11:13:40.981634 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:40.981422 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9314229f-a3c5-4809-8129-b06de0b52127" containerName="prom-label-proxy" Jul 07 11:13:40.981634 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:40.981427 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="9314229f-a3c5-4809-8129-b06de0b52127" containerName="prom-label-proxy" Jul 07 11:13:40.981634 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:40.981433 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9314229f-a3c5-4809-8129-b06de0b52127" containerName="config-reloader" Jul 07 11:13:40.981634 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:40.981438 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="9314229f-a3c5-4809-8129-b06de0b52127" containerName="config-reloader" Jul 07 11:13:40.981634 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:40.981443 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9314229f-a3c5-4809-8129-b06de0b52127" containerName="kube-rbac-proxy" Jul 07 11:13:40.981634 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:40.981447 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="9314229f-a3c5-4809-8129-b06de0b52127" containerName="kube-rbac-proxy" Jul 07 11:13:40.981634 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:40.981495 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="9314229f-a3c5-4809-8129-b06de0b52127" containerName="alertmanager" Jul 07 11:13:40.981634 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:40.981502 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="9314229f-a3c5-4809-8129-b06de0b52127" containerName="kube-rbac-proxy-web" Jul 07 11:13:40.981634 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:40.981512 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="9314229f-a3c5-4809-8129-b06de0b52127" containerName="kube-rbac-proxy" Jul 07 11:13:40.981634 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:40.981518 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="9314229f-a3c5-4809-8129-b06de0b52127" containerName="config-reloader" Jul 07 11:13:40.981634 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:40.981525 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="9314229f-a3c5-4809-8129-b06de0b52127" containerName="kube-rbac-proxy-metric" Jul 07 11:13:40.981634 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:40.981531 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="9314229f-a3c5-4809-8129-b06de0b52127" containerName="prom-label-proxy" Jul 07 11:13:40.986513 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:40.986492 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:13:40.989865 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:40.989409 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 07 11:13:40.989865 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:40.989465 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 07 11:13:40.989865 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:40.989790 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 07 11:13:40.990142 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:40.990117 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 07 11:13:40.990200 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:40.990185 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 07 11:13:40.990802 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:40.990666 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 07 11:13:40.991953 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:40.991932 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 07 11:13:40.992549 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:40.992383 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-nlp2c\"" Jul 07 11:13:40.992649 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:40.992604 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 07 11:13:40.998273 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:40.998100 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 07 11:13:40.999162 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:40.999079 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 07 11:13:41.043661 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:41.043638 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/9799c267-e325-4239-94d3-023d50dd3fb3-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"9799c267-e325-4239-94d3-023d50dd3fb3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:13:41.043758 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:41.043664 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/9799c267-e325-4239-94d3-023d50dd3fb3-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"9799c267-e325-4239-94d3-023d50dd3fb3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:13:41.043758 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:41.043686 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9799c267-e325-4239-94d3-023d50dd3fb3-tls-assets\") pod \"alertmanager-main-0\" (UID: \"9799c267-e325-4239-94d3-023d50dd3fb3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:13:41.043758 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:41.043710 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9799c267-e325-4239-94d3-023d50dd3fb3-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"9799c267-e325-4239-94d3-023d50dd3fb3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:13:41.043758 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:41.043727 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9799c267-e325-4239-94d3-023d50dd3fb3-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"9799c267-e325-4239-94d3-023d50dd3fb3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:13:41.043989 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:41.043775 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/9799c267-e325-4239-94d3-023d50dd3fb3-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"9799c267-e325-4239-94d3-023d50dd3fb3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:13:41.043989 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:41.043791 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/9799c267-e325-4239-94d3-023d50dd3fb3-config-volume\") pod \"alertmanager-main-0\" (UID: \"9799c267-e325-4239-94d3-023d50dd3fb3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:13:41.043989 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:41.043805 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-msj4q\" (UniqueName: \"kubernetes.io/projected/9799c267-e325-4239-94d3-023d50dd3fb3-kube-api-access-msj4q\") pod \"alertmanager-main-0\" (UID: \"9799c267-e325-4239-94d3-023d50dd3fb3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:13:41.043989 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:41.043883 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/9799c267-e325-4239-94d3-023d50dd3fb3-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"9799c267-e325-4239-94d3-023d50dd3fb3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:13:41.043989 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:41.043953 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9799c267-e325-4239-94d3-023d50dd3fb3-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"9799c267-e325-4239-94d3-023d50dd3fb3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:13:41.043989 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:41.043981 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9799c267-e325-4239-94d3-023d50dd3fb3-web-config\") pod \"alertmanager-main-0\" (UID: \"9799c267-e325-4239-94d3-023d50dd3fb3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:13:41.044213 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:41.044005 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9799c267-e325-4239-94d3-023d50dd3fb3-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"9799c267-e325-4239-94d3-023d50dd3fb3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:13:41.044213 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:41.044042 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9799c267-e325-4239-94d3-023d50dd3fb3-config-out\") pod \"alertmanager-main-0\" (UID: \"9799c267-e325-4239-94d3-023d50dd3fb3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:13:41.144498 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:41.144463 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/9799c267-e325-4239-94d3-023d50dd3fb3-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"9799c267-e325-4239-94d3-023d50dd3fb3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:13:41.144662 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:41.144508 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9799c267-e325-4239-94d3-023d50dd3fb3-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"9799c267-e325-4239-94d3-023d50dd3fb3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:13:41.144662 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:41.144532 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9799c267-e325-4239-94d3-023d50dd3fb3-web-config\") pod \"alertmanager-main-0\" (UID: \"9799c267-e325-4239-94d3-023d50dd3fb3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:13:41.144662 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:41.144566 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9799c267-e325-4239-94d3-023d50dd3fb3-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"9799c267-e325-4239-94d3-023d50dd3fb3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:13:41.144662 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:41.144604 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9799c267-e325-4239-94d3-023d50dd3fb3-config-out\") pod \"alertmanager-main-0\" (UID: \"9799c267-e325-4239-94d3-023d50dd3fb3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:13:41.144662 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:41.144639 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/9799c267-e325-4239-94d3-023d50dd3fb3-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"9799c267-e325-4239-94d3-023d50dd3fb3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:13:41.144885 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:41.144764 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/9799c267-e325-4239-94d3-023d50dd3fb3-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"9799c267-e325-4239-94d3-023d50dd3fb3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:13:41.144885 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:41.144809 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9799c267-e325-4239-94d3-023d50dd3fb3-tls-assets\") pod \"alertmanager-main-0\" (UID: \"9799c267-e325-4239-94d3-023d50dd3fb3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:13:41.144885 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:41.144860 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9799c267-e325-4239-94d3-023d50dd3fb3-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"9799c267-e325-4239-94d3-023d50dd3fb3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:13:41.145061 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:41.144892 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9799c267-e325-4239-94d3-023d50dd3fb3-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"9799c267-e325-4239-94d3-023d50dd3fb3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:13:41.145061 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:41.144967 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/9799c267-e325-4239-94d3-023d50dd3fb3-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"9799c267-e325-4239-94d3-023d50dd3fb3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:13:41.145061 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:41.144998 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/9799c267-e325-4239-94d3-023d50dd3fb3-config-volume\") pod \"alertmanager-main-0\" (UID: \"9799c267-e325-4239-94d3-023d50dd3fb3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:13:41.145061 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:41.145020 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-msj4q\" (UniqueName: \"kubernetes.io/projected/9799c267-e325-4239-94d3-023d50dd3fb3-kube-api-access-msj4q\") pod \"alertmanager-main-0\" (UID: \"9799c267-e325-4239-94d3-023d50dd3fb3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:13:41.145361 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:41.145123 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/9799c267-e325-4239-94d3-023d50dd3fb3-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"9799c267-e325-4239-94d3-023d50dd3fb3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:13:41.145495 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:41.145466 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9799c267-e325-4239-94d3-023d50dd3fb3-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"9799c267-e325-4239-94d3-023d50dd3fb3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:13:41.146381 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:41.146355 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9799c267-e325-4239-94d3-023d50dd3fb3-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"9799c267-e325-4239-94d3-023d50dd3fb3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:13:41.147636 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:41.147609 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9799c267-e325-4239-94d3-023d50dd3fb3-web-config\") pod \"alertmanager-main-0\" (UID: \"9799c267-e325-4239-94d3-023d50dd3fb3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:13:41.147636 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:41.147629 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9799c267-e325-4239-94d3-023d50dd3fb3-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"9799c267-e325-4239-94d3-023d50dd3fb3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:13:41.147901 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:41.147877 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9799c267-e325-4239-94d3-023d50dd3fb3-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"9799c267-e325-4239-94d3-023d50dd3fb3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:13:41.148011 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:41.147895 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/9799c267-e325-4239-94d3-023d50dd3fb3-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"9799c267-e325-4239-94d3-023d50dd3fb3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:13:41.148011 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:41.147963 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9799c267-e325-4239-94d3-023d50dd3fb3-config-out\") pod \"alertmanager-main-0\" (UID: \"9799c267-e325-4239-94d3-023d50dd3fb3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:13:41.148112 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:41.148029 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/9799c267-e325-4239-94d3-023d50dd3fb3-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"9799c267-e325-4239-94d3-023d50dd3fb3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:13:41.148151 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:41.148122 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/9799c267-e325-4239-94d3-023d50dd3fb3-config-volume\") pod \"alertmanager-main-0\" (UID: \"9799c267-e325-4239-94d3-023d50dd3fb3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:13:41.148731 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:41.148715 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/9799c267-e325-4239-94d3-023d50dd3fb3-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"9799c267-e325-4239-94d3-023d50dd3fb3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:13:41.149442 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:41.149427 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9799c267-e325-4239-94d3-023d50dd3fb3-tls-assets\") pod \"alertmanager-main-0\" (UID: \"9799c267-e325-4239-94d3-023d50dd3fb3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:13:41.153828 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:41.153805 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-msj4q\" (UniqueName: \"kubernetes.io/projected/9799c267-e325-4239-94d3-023d50dd3fb3-kube-api-access-msj4q\") pod \"alertmanager-main-0\" (UID: \"9799c267-e325-4239-94d3-023d50dd3fb3\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:13:41.300992 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:41.300938 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 07 11:13:41.463115 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:13:41.463090 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9799c267_e325_4239_94d3_023d50dd3fb3.slice/crio-fcc2a55d62005385c5719efde4efe6dfa62cd0016f864cad87586797852a4213 WatchSource:0}: Error finding container fcc2a55d62005385c5719efde4efe6dfa62cd0016f864cad87586797852a4213: Status 404 returned error can't find the container with id fcc2a55d62005385c5719efde4efe6dfa62cd0016f864cad87586797852a4213 Jul 07 11:13:41.463222 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:41.463198 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 07 11:13:41.631055 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:41.631028 2562 generic.go:356] "Generic (PLEG): container finished" podID="9799c267-e325-4239-94d3-023d50dd3fb3" containerID="386c5e05ab3dc04671ecc74d7f3e1cb6ece8f82f64f7f57ac8bcf426e79b067d" exitCode=0 Jul 07 11:13:41.631341 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:41.631097 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9799c267-e325-4239-94d3-023d50dd3fb3","Type":"ContainerDied","Data":"386c5e05ab3dc04671ecc74d7f3e1cb6ece8f82f64f7f57ac8bcf426e79b067d"} Jul 07 11:13:41.631341 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:41.631170 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9799c267-e325-4239-94d3-023d50dd3fb3","Type":"ContainerStarted","Data":"fcc2a55d62005385c5719efde4efe6dfa62cd0016f864cad87586797852a4213"} Jul 07 11:13:41.766729 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:41.766704 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9314229f-a3c5-4809-8129-b06de0b52127" path="/var/lib/kubelet/pods/9314229f-a3c5-4809-8129-b06de0b52127/volumes" Jul 07 11:13:42.639015 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:42.638925 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9799c267-e325-4239-94d3-023d50dd3fb3","Type":"ContainerStarted","Data":"64fa7a27a5590c58b4fec5592545c6f19375f73adbbf1c312ccbf5c078c24fc6"} Jul 07 11:13:42.639015 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:42.639004 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9799c267-e325-4239-94d3-023d50dd3fb3","Type":"ContainerStarted","Data":"6a067e9e4ec0dd88840d835ddce6db9fd78972b6a206593e8a820770d95ef6c4"} Jul 07 11:13:42.639475 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:42.639045 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9799c267-e325-4239-94d3-023d50dd3fb3","Type":"ContainerStarted","Data":"3df1d63fea44105038855ec69f707e66c694810a1f0824553dd25bf560cbd22b"} Jul 07 11:13:42.639475 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:42.639082 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9799c267-e325-4239-94d3-023d50dd3fb3","Type":"ContainerStarted","Data":"49d1f091d6d5684401b3e960b7ed0c5b3868a4641fcf549bafd486c0e7133d09"} Jul 07 11:13:42.639475 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:42.639116 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9799c267-e325-4239-94d3-023d50dd3fb3","Type":"ContainerStarted","Data":"e83e129e59ad7282837e184af339e0be1f4f42789c6d7e9b8589f2828748c15e"} Jul 07 11:13:42.639475 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:42.639150 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9799c267-e325-4239-94d3-023d50dd3fb3","Type":"ContainerStarted","Data":"7883ea7172f140e7aac05770ceb4c939a5d7486bdd2417b3b6faafbf0c7bb146"} Jul 07 11:13:42.668877 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:42.668834 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.653786176 podStartE2EDuration="2.668818969s" totalImagesPullingTime="0s" totalInitContainerRuntime="15.032793ms" isStatefulPod=false podCreationTimestamp="2026-07-07 11:13:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:13:42.667991872 +0000 UTC m=+147.081861050" watchObservedRunningTime="2026-07-07 11:13:42.668818969 +0000 UTC m=+147.082688128" Jul 07 11:13:43.554159 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:43.554118 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 07 11:13:43.555083 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:43.555048 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="c088023f-86df-4531-be68-1a21bad47ae6" containerName="prometheus" containerID="cri-o://e98438c43873adcbb713a7890ec5d37b1f1619423ae36cc69c9dc54490253c68" gracePeriod=600 Jul 07 11:13:43.555644 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:43.555419 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="c088023f-86df-4531-be68-1a21bad47ae6" containerName="kube-rbac-proxy-thanos" containerID="cri-o://39d4460ed91020e9f75de9cec8cd0334ef6c845ef86544427f906b3b7d1eb463" gracePeriod=600 Jul 07 11:13:43.555965 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:43.555445 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="c088023f-86df-4531-be68-1a21bad47ae6" containerName="kube-rbac-proxy-web" containerID="cri-o://9bddc02f54e220f34e3c00d4af2883a1576cd50ea4fcba3ed0450e695db4105d" gracePeriod=600 Jul 07 11:13:43.556088 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:43.555466 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="c088023f-86df-4531-be68-1a21bad47ae6" containerName="kube-rbac-proxy" containerID="cri-o://06bf9b5a2354ff69ec76d04926974872d40a7e36cedbb69c51792d0c84383954" gracePeriod=600 Jul 07 11:13:43.556529 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:43.555480 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="c088023f-86df-4531-be68-1a21bad47ae6" containerName="thanos-sidecar" containerID="cri-o://f7f965b05c081819965360a985cc73476b8f1b08b69eeff76dfd2d119f816822" gracePeriod=600 Jul 07 11:13:43.556529 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:43.555496 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="c088023f-86df-4531-be68-1a21bad47ae6" containerName="config-reloader" containerID="cri-o://69ecb11f604d74c323cefb8de01623ef2a58f38b7b2ba3a9214143b2b494ec6d" gracePeriod=600 Jul 07 11:13:43.818447 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:43.818415 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:13:43.871373 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:43.871329 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/c088023f-86df-4531-be68-1a21bad47ae6-config\") pod \"c088023f-86df-4531-be68-1a21bad47ae6\" (UID: \"c088023f-86df-4531-be68-1a21bad47ae6\") " Jul 07 11:13:43.871541 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:43.871385 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c088023f-86df-4531-be68-1a21bad47ae6-tls-assets\") pod \"c088023f-86df-4531-be68-1a21bad47ae6\" (UID: \"c088023f-86df-4531-be68-1a21bad47ae6\") " Jul 07 11:13:43.871541 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:43.871415 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/c088023f-86df-4531-be68-1a21bad47ae6-thanos-prometheus-http-client-file\") pod \"c088023f-86df-4531-be68-1a21bad47ae6\" (UID: \"c088023f-86df-4531-be68-1a21bad47ae6\") " Jul 07 11:13:43.871541 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:43.871456 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c088023f-86df-4531-be68-1a21bad47ae6-prometheus-trusted-ca-bundle\") pod \"c088023f-86df-4531-be68-1a21bad47ae6\" (UID: \"c088023f-86df-4531-be68-1a21bad47ae6\") " Jul 07 11:13:43.871541 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:43.871489 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c088023f-86df-4531-be68-1a21bad47ae6-secret-grpc-tls\") pod \"c088023f-86df-4531-be68-1a21bad47ae6\" (UID: \"c088023f-86df-4531-be68-1a21bad47ae6\") " Jul 07 11:13:43.871541 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:43.871518 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/c088023f-86df-4531-be68-1a21bad47ae6-prometheus-k8s-db\") pod \"c088023f-86df-4531-be68-1a21bad47ae6\" (UID: \"c088023f-86df-4531-be68-1a21bad47ae6\") " Jul 07 11:13:43.871768 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:43.871543 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c088023f-86df-4531-be68-1a21bad47ae6-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"c088023f-86df-4531-be68-1a21bad47ae6\" (UID: \"c088023f-86df-4531-be68-1a21bad47ae6\") " Jul 07 11:13:43.871768 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:43.871580 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/c088023f-86df-4531-be68-1a21bad47ae6-prometheus-k8s-rulefiles-2\") pod \"c088023f-86df-4531-be68-1a21bad47ae6\" (UID: \"c088023f-86df-4531-be68-1a21bad47ae6\") " Jul 07 11:13:43.871768 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:43.871612 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c088023f-86df-4531-be68-1a21bad47ae6-configmap-serving-certs-ca-bundle\") pod \"c088023f-86df-4531-be68-1a21bad47ae6\" (UID: \"c088023f-86df-4531-be68-1a21bad47ae6\") " Jul 07 11:13:43.871768 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:43.871652 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c088023f-86df-4531-be68-1a21bad47ae6-secret-kube-rbac-proxy\") pod \"c088023f-86df-4531-be68-1a21bad47ae6\" (UID: \"c088023f-86df-4531-be68-1a21bad47ae6\") " Jul 07 11:13:43.871768 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:43.871675 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c088023f-86df-4531-be68-1a21bad47ae6-web-config\") pod \"c088023f-86df-4531-be68-1a21bad47ae6\" (UID: \"c088023f-86df-4531-be68-1a21bad47ae6\") " Jul 07 11:13:43.871768 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:43.871702 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/c088023f-86df-4531-be68-1a21bad47ae6-secret-metrics-client-certs\") pod \"c088023f-86df-4531-be68-1a21bad47ae6\" (UID: \"c088023f-86df-4531-be68-1a21bad47ae6\") " Jul 07 11:13:43.871768 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:43.871732 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c088023f-86df-4531-be68-1a21bad47ae6-configmap-metrics-client-ca\") pod \"c088023f-86df-4531-be68-1a21bad47ae6\" (UID: \"c088023f-86df-4531-be68-1a21bad47ae6\") " Jul 07 11:13:43.872071 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:43.871779 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/c088023f-86df-4531-be68-1a21bad47ae6-prometheus-k8s-rulefiles-1\") pod \"c088023f-86df-4531-be68-1a21bad47ae6\" (UID: \"c088023f-86df-4531-be68-1a21bad47ae6\") " Jul 07 11:13:43.872071 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:43.871815 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c088023f-86df-4531-be68-1a21bad47ae6-configmap-kubelet-serving-ca-bundle\") pod \"c088023f-86df-4531-be68-1a21bad47ae6\" (UID: \"c088023f-86df-4531-be68-1a21bad47ae6\") " Jul 07 11:13:43.872071 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:43.871866 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/c088023f-86df-4531-be68-1a21bad47ae6-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"c088023f-86df-4531-be68-1a21bad47ae6\" (UID: \"c088023f-86df-4531-be68-1a21bad47ae6\") " Jul 07 11:13:43.872071 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:43.871892 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-npr2b\" (UniqueName: \"kubernetes.io/projected/c088023f-86df-4531-be68-1a21bad47ae6-kube-api-access-npr2b\") pod \"c088023f-86df-4531-be68-1a21bad47ae6\" (UID: \"c088023f-86df-4531-be68-1a21bad47ae6\") " Jul 07 11:13:43.872071 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:43.872057 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/c088023f-86df-4531-be68-1a21bad47ae6-prometheus-k8s-rulefiles-0\") pod \"c088023f-86df-4531-be68-1a21bad47ae6\" (UID: \"c088023f-86df-4531-be68-1a21bad47ae6\") " Jul 07 11:13:43.872318 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:43.872110 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/c088023f-86df-4531-be68-1a21bad47ae6-secret-prometheus-k8s-tls\") pod \"c088023f-86df-4531-be68-1a21bad47ae6\" (UID: \"c088023f-86df-4531-be68-1a21bad47ae6\") " Jul 07 11:13:43.872318 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:43.872142 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c088023f-86df-4531-be68-1a21bad47ae6-config-out\") pod \"c088023f-86df-4531-be68-1a21bad47ae6\" (UID: \"c088023f-86df-4531-be68-1a21bad47ae6\") " Jul 07 11:13:43.872425 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:43.872375 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c088023f-86df-4531-be68-1a21bad47ae6-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "c088023f-86df-4531-be68-1a21bad47ae6" (UID: "c088023f-86df-4531-be68-1a21bad47ae6"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 11:13:43.872483 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:43.872460 2562 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c088023f-86df-4531-be68-1a21bad47ae6-prometheus-trusted-ca-bundle\") on node \"ip-10-0-136-15.ec2.internal\" DevicePath \"\"" Jul 07 11:13:43.872740 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:43.872711 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c088023f-86df-4531-be68-1a21bad47ae6-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "c088023f-86df-4531-be68-1a21bad47ae6" (UID: "c088023f-86df-4531-be68-1a21bad47ae6"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 11:13:43.873052 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:43.873023 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c088023f-86df-4531-be68-1a21bad47ae6-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "c088023f-86df-4531-be68-1a21bad47ae6" (UID: "c088023f-86df-4531-be68-1a21bad47ae6"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 11:13:43.873139 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:43.873046 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c088023f-86df-4531-be68-1a21bad47ae6-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "c088023f-86df-4531-be68-1a21bad47ae6" (UID: "c088023f-86df-4531-be68-1a21bad47ae6"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 11:13:43.874639 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:43.874393 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c088023f-86df-4531-be68-1a21bad47ae6-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "c088023f-86df-4531-be68-1a21bad47ae6" (UID: "c088023f-86df-4531-be68-1a21bad47ae6"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 11:13:43.875798 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:43.875767 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c088023f-86df-4531-be68-1a21bad47ae6-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "c088023f-86df-4531-be68-1a21bad47ae6" (UID: "c088023f-86df-4531-be68-1a21bad47ae6"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 11:13:43.875947 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:43.875926 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c088023f-86df-4531-be68-1a21bad47ae6-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "c088023f-86df-4531-be68-1a21bad47ae6" (UID: "c088023f-86df-4531-be68-1a21bad47ae6"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 11:13:43.877234 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:43.877206 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c088023f-86df-4531-be68-1a21bad47ae6-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "c088023f-86df-4531-be68-1a21bad47ae6" (UID: "c088023f-86df-4531-be68-1a21bad47ae6"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 11:13:43.877486 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:43.877457 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c088023f-86df-4531-be68-1a21bad47ae6-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "c088023f-86df-4531-be68-1a21bad47ae6" (UID: "c088023f-86df-4531-be68-1a21bad47ae6"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 11:13:43.877641 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:43.877618 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c088023f-86df-4531-be68-1a21bad47ae6-config-out" (OuterVolumeSpecName: "config-out") pod "c088023f-86df-4531-be68-1a21bad47ae6" (UID: "c088023f-86df-4531-be68-1a21bad47ae6"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 11:13:43.879483 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:43.879156 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c088023f-86df-4531-be68-1a21bad47ae6-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "c088023f-86df-4531-be68-1a21bad47ae6" (UID: "c088023f-86df-4531-be68-1a21bad47ae6"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 11:13:43.879483 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:43.879204 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c088023f-86df-4531-be68-1a21bad47ae6-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "c088023f-86df-4531-be68-1a21bad47ae6" (UID: "c088023f-86df-4531-be68-1a21bad47ae6"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 11:13:43.879483 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:43.879407 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c088023f-86df-4531-be68-1a21bad47ae6-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "c088023f-86df-4531-be68-1a21bad47ae6" (UID: "c088023f-86df-4531-be68-1a21bad47ae6"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 11:13:43.879711 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:43.879549 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c088023f-86df-4531-be68-1a21bad47ae6-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "c088023f-86df-4531-be68-1a21bad47ae6" (UID: "c088023f-86df-4531-be68-1a21bad47ae6"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 11:13:43.879711 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:43.879646 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c088023f-86df-4531-be68-1a21bad47ae6-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "c088023f-86df-4531-be68-1a21bad47ae6" (UID: "c088023f-86df-4531-be68-1a21bad47ae6"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 11:13:43.879822 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:43.879706 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c088023f-86df-4531-be68-1a21bad47ae6-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "c088023f-86df-4531-be68-1a21bad47ae6" (UID: "c088023f-86df-4531-be68-1a21bad47ae6"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 11:13:43.879822 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:43.879793 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c088023f-86df-4531-be68-1a21bad47ae6-config" (OuterVolumeSpecName: "config") pod "c088023f-86df-4531-be68-1a21bad47ae6" (UID: "c088023f-86df-4531-be68-1a21bad47ae6"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 11:13:43.880017 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:43.879865 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c088023f-86df-4531-be68-1a21bad47ae6-kube-api-access-npr2b" (OuterVolumeSpecName: "kube-api-access-npr2b") pod "c088023f-86df-4531-be68-1a21bad47ae6" (UID: "c088023f-86df-4531-be68-1a21bad47ae6"). InnerVolumeSpecName "kube-api-access-npr2b". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 11:13:43.880017 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:43.879879 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c088023f-86df-4531-be68-1a21bad47ae6-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "c088023f-86df-4531-be68-1a21bad47ae6" (UID: "c088023f-86df-4531-be68-1a21bad47ae6"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 11:13:43.890958 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:43.890933 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c088023f-86df-4531-be68-1a21bad47ae6-web-config" (OuterVolumeSpecName: "web-config") pod "c088023f-86df-4531-be68-1a21bad47ae6" (UID: "c088023f-86df-4531-be68-1a21bad47ae6"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 11:13:43.973331 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:43.973301 2562 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c088023f-86df-4531-be68-1a21bad47ae6-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-136-15.ec2.internal\" DevicePath \"\"" Jul 07 11:13:43.973331 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:43.973328 2562 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/c088023f-86df-4531-be68-1a21bad47ae6-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-136-15.ec2.internal\" DevicePath \"\"" Jul 07 11:13:43.973493 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:43.973344 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-npr2b\" (UniqueName: \"kubernetes.io/projected/c088023f-86df-4531-be68-1a21bad47ae6-kube-api-access-npr2b\") on node \"ip-10-0-136-15.ec2.internal\" DevicePath \"\"" Jul 07 11:13:43.973493 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:43.973358 2562 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/c088023f-86df-4531-be68-1a21bad47ae6-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-136-15.ec2.internal\" DevicePath \"\"" Jul 07 11:13:43.973493 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:43.973375 2562 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/c088023f-86df-4531-be68-1a21bad47ae6-secret-prometheus-k8s-tls\") on node \"ip-10-0-136-15.ec2.internal\" DevicePath \"\"" Jul 07 11:13:43.973493 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:43.973390 2562 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c088023f-86df-4531-be68-1a21bad47ae6-config-out\") on node \"ip-10-0-136-15.ec2.internal\" DevicePath \"\"" Jul 07 11:13:43.973493 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:43.973404 2562 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/c088023f-86df-4531-be68-1a21bad47ae6-config\") on node \"ip-10-0-136-15.ec2.internal\" DevicePath \"\"" Jul 07 11:13:43.973493 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:43.973417 2562 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c088023f-86df-4531-be68-1a21bad47ae6-tls-assets\") on node \"ip-10-0-136-15.ec2.internal\" DevicePath \"\"" Jul 07 11:13:43.973493 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:43.973430 2562 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/c088023f-86df-4531-be68-1a21bad47ae6-thanos-prometheus-http-client-file\") on node \"ip-10-0-136-15.ec2.internal\" DevicePath \"\"" Jul 07 11:13:43.973493 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:43.973444 2562 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c088023f-86df-4531-be68-1a21bad47ae6-secret-grpc-tls\") on node \"ip-10-0-136-15.ec2.internal\" DevicePath \"\"" Jul 07 11:13:43.973493 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:43.973458 2562 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/c088023f-86df-4531-be68-1a21bad47ae6-prometheus-k8s-db\") on node \"ip-10-0-136-15.ec2.internal\" DevicePath \"\"" Jul 07 11:13:43.973493 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:43.973473 2562 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c088023f-86df-4531-be68-1a21bad47ae6-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-136-15.ec2.internal\" DevicePath \"\"" Jul 07 11:13:43.973493 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:43.973488 2562 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/c088023f-86df-4531-be68-1a21bad47ae6-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-136-15.ec2.internal\" DevicePath \"\"" Jul 07 11:13:43.975503 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:43.973503 2562 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c088023f-86df-4531-be68-1a21bad47ae6-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-136-15.ec2.internal\" DevicePath \"\"" Jul 07 11:13:43.975503 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:43.973519 2562 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c088023f-86df-4531-be68-1a21bad47ae6-secret-kube-rbac-proxy\") on node \"ip-10-0-136-15.ec2.internal\" DevicePath \"\"" Jul 07 11:13:43.975503 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:43.973534 2562 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c088023f-86df-4531-be68-1a21bad47ae6-web-config\") on node \"ip-10-0-136-15.ec2.internal\" DevicePath \"\"" Jul 07 11:13:43.975503 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:43.973548 2562 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/c088023f-86df-4531-be68-1a21bad47ae6-secret-metrics-client-certs\") on node \"ip-10-0-136-15.ec2.internal\" DevicePath \"\"" Jul 07 11:13:43.975503 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:43.973561 2562 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c088023f-86df-4531-be68-1a21bad47ae6-configmap-metrics-client-ca\") on node \"ip-10-0-136-15.ec2.internal\" DevicePath \"\"" Jul 07 11:13:43.975503 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:43.973578 2562 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/c088023f-86df-4531-be68-1a21bad47ae6-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-136-15.ec2.internal\" DevicePath \"\"" Jul 07 11:13:44.651506 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.651473 2562 generic.go:356] "Generic (PLEG): container finished" podID="c088023f-86df-4531-be68-1a21bad47ae6" containerID="39d4460ed91020e9f75de9cec8cd0334ef6c845ef86544427f906b3b7d1eb463" exitCode=0 Jul 07 11:13:44.651506 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.651498 2562 generic.go:356] "Generic (PLEG): container finished" podID="c088023f-86df-4531-be68-1a21bad47ae6" containerID="06bf9b5a2354ff69ec76d04926974872d40a7e36cedbb69c51792d0c84383954" exitCode=0 Jul 07 11:13:44.651506 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.651505 2562 generic.go:356] "Generic (PLEG): container finished" podID="c088023f-86df-4531-be68-1a21bad47ae6" containerID="9bddc02f54e220f34e3c00d4af2883a1576cd50ea4fcba3ed0450e695db4105d" exitCode=0 Jul 07 11:13:44.651506 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.651510 2562 generic.go:356] "Generic (PLEG): container finished" podID="c088023f-86df-4531-be68-1a21bad47ae6" containerID="f7f965b05c081819965360a985cc73476b8f1b08b69eeff76dfd2d119f816822" exitCode=0 Jul 07 11:13:44.651506 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.651515 2562 generic.go:356] "Generic (PLEG): container finished" podID="c088023f-86df-4531-be68-1a21bad47ae6" containerID="69ecb11f604d74c323cefb8de01623ef2a58f38b7b2ba3a9214143b2b494ec6d" exitCode=0 Jul 07 11:13:44.651855 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.651521 2562 generic.go:356] "Generic (PLEG): container finished" podID="c088023f-86df-4531-be68-1a21bad47ae6" containerID="e98438c43873adcbb713a7890ec5d37b1f1619423ae36cc69c9dc54490253c68" exitCode=0 Jul 07 11:13:44.651855 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.651528 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c088023f-86df-4531-be68-1a21bad47ae6","Type":"ContainerDied","Data":"39d4460ed91020e9f75de9cec8cd0334ef6c845ef86544427f906b3b7d1eb463"} Jul 07 11:13:44.651855 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.651571 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:13:44.651855 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.651596 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c088023f-86df-4531-be68-1a21bad47ae6","Type":"ContainerDied","Data":"06bf9b5a2354ff69ec76d04926974872d40a7e36cedbb69c51792d0c84383954"} Jul 07 11:13:44.651855 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.651626 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c088023f-86df-4531-be68-1a21bad47ae6","Type":"ContainerDied","Data":"9bddc02f54e220f34e3c00d4af2883a1576cd50ea4fcba3ed0450e695db4105d"} Jul 07 11:13:44.651855 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.651651 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c088023f-86df-4531-be68-1a21bad47ae6","Type":"ContainerDied","Data":"f7f965b05c081819965360a985cc73476b8f1b08b69eeff76dfd2d119f816822"} Jul 07 11:13:44.651855 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.651663 2562 scope.go:117] "RemoveContainer" containerID="39d4460ed91020e9f75de9cec8cd0334ef6c845ef86544427f906b3b7d1eb463" Jul 07 11:13:44.651855 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.651679 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c088023f-86df-4531-be68-1a21bad47ae6","Type":"ContainerDied","Data":"69ecb11f604d74c323cefb8de01623ef2a58f38b7b2ba3a9214143b2b494ec6d"} Jul 07 11:13:44.651855 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.651704 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c088023f-86df-4531-be68-1a21bad47ae6","Type":"ContainerDied","Data":"e98438c43873adcbb713a7890ec5d37b1f1619423ae36cc69c9dc54490253c68"} Jul 07 11:13:44.651855 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.651727 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c088023f-86df-4531-be68-1a21bad47ae6","Type":"ContainerDied","Data":"0c75b9c0bb1bcfd0f25e44c6d258242900fc2539c7fb56e3597df424f567be22"} Jul 07 11:13:44.660273 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.660258 2562 scope.go:117] "RemoveContainer" containerID="06bf9b5a2354ff69ec76d04926974872d40a7e36cedbb69c51792d0c84383954" Jul 07 11:13:44.667233 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.667209 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:13:44.667641 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.667626 2562 scope.go:117] "RemoveContainer" containerID="9bddc02f54e220f34e3c00d4af2883a1576cd50ea4fcba3ed0450e695db4105d" Jul 07 11:13:44.673958 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.673941 2562 scope.go:117] "RemoveContainer" containerID="f7f965b05c081819965360a985cc73476b8f1b08b69eeff76dfd2d119f816822" Jul 07 11:13:44.679213 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.679189 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 07 11:13:44.680975 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.680960 2562 scope.go:117] "RemoveContainer" containerID="69ecb11f604d74c323cefb8de01623ef2a58f38b7b2ba3a9214143b2b494ec6d" Jul 07 11:13:44.683962 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.683940 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 07 11:13:44.687824 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.687810 2562 scope.go:117] "RemoveContainer" containerID="e98438c43873adcbb713a7890ec5d37b1f1619423ae36cc69c9dc54490253c68" Jul 07 11:13:44.694705 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.694689 2562 scope.go:117] "RemoveContainer" containerID="540adf46eb218874649adb7703958545843d4594f01f897d43762bc2cf395532" Jul 07 11:13:44.701049 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.701028 2562 scope.go:117] "RemoveContainer" containerID="39d4460ed91020e9f75de9cec8cd0334ef6c845ef86544427f906b3b7d1eb463" Jul 07 11:13:44.701303 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:13:44.701287 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"39d4460ed91020e9f75de9cec8cd0334ef6c845ef86544427f906b3b7d1eb463\": container with ID starting with 39d4460ed91020e9f75de9cec8cd0334ef6c845ef86544427f906b3b7d1eb463 not found: ID does not exist" containerID="39d4460ed91020e9f75de9cec8cd0334ef6c845ef86544427f906b3b7d1eb463" Jul 07 11:13:44.701350 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.701310 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"39d4460ed91020e9f75de9cec8cd0334ef6c845ef86544427f906b3b7d1eb463"} err="failed to get container status \"39d4460ed91020e9f75de9cec8cd0334ef6c845ef86544427f906b3b7d1eb463\": rpc error: code = NotFound desc = could not find container \"39d4460ed91020e9f75de9cec8cd0334ef6c845ef86544427f906b3b7d1eb463\": container with ID starting with 39d4460ed91020e9f75de9cec8cd0334ef6c845ef86544427f906b3b7d1eb463 not found: ID does not exist" Jul 07 11:13:44.701350 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.701326 2562 scope.go:117] "RemoveContainer" containerID="06bf9b5a2354ff69ec76d04926974872d40a7e36cedbb69c51792d0c84383954" Jul 07 11:13:44.701541 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:13:44.701525 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"06bf9b5a2354ff69ec76d04926974872d40a7e36cedbb69c51792d0c84383954\": container with ID starting with 06bf9b5a2354ff69ec76d04926974872d40a7e36cedbb69c51792d0c84383954 not found: ID does not exist" containerID="06bf9b5a2354ff69ec76d04926974872d40a7e36cedbb69c51792d0c84383954" Jul 07 11:13:44.701580 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.701545 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"06bf9b5a2354ff69ec76d04926974872d40a7e36cedbb69c51792d0c84383954"} err="failed to get container status \"06bf9b5a2354ff69ec76d04926974872d40a7e36cedbb69c51792d0c84383954\": rpc error: code = NotFound desc = could not find container \"06bf9b5a2354ff69ec76d04926974872d40a7e36cedbb69c51792d0c84383954\": container with ID starting with 06bf9b5a2354ff69ec76d04926974872d40a7e36cedbb69c51792d0c84383954 not found: ID does not exist" Jul 07 11:13:44.701580 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.701557 2562 scope.go:117] "RemoveContainer" containerID="9bddc02f54e220f34e3c00d4af2883a1576cd50ea4fcba3ed0450e695db4105d" Jul 07 11:13:44.701740 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:13:44.701726 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9bddc02f54e220f34e3c00d4af2883a1576cd50ea4fcba3ed0450e695db4105d\": container with ID starting with 9bddc02f54e220f34e3c00d4af2883a1576cd50ea4fcba3ed0450e695db4105d not found: ID does not exist" containerID="9bddc02f54e220f34e3c00d4af2883a1576cd50ea4fcba3ed0450e695db4105d" Jul 07 11:13:44.701808 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.701742 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9bddc02f54e220f34e3c00d4af2883a1576cd50ea4fcba3ed0450e695db4105d"} err="failed to get container status \"9bddc02f54e220f34e3c00d4af2883a1576cd50ea4fcba3ed0450e695db4105d\": rpc error: code = NotFound desc = could not find container \"9bddc02f54e220f34e3c00d4af2883a1576cd50ea4fcba3ed0450e695db4105d\": container with ID starting with 9bddc02f54e220f34e3c00d4af2883a1576cd50ea4fcba3ed0450e695db4105d not found: ID does not exist" Jul 07 11:13:44.701808 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.701753 2562 scope.go:117] "RemoveContainer" containerID="f7f965b05c081819965360a985cc73476b8f1b08b69eeff76dfd2d119f816822" Jul 07 11:13:44.701965 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:13:44.701949 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f7f965b05c081819965360a985cc73476b8f1b08b69eeff76dfd2d119f816822\": container with ID starting with f7f965b05c081819965360a985cc73476b8f1b08b69eeff76dfd2d119f816822 not found: ID does not exist" containerID="f7f965b05c081819965360a985cc73476b8f1b08b69eeff76dfd2d119f816822" Jul 07 11:13:44.702013 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.701969 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f7f965b05c081819965360a985cc73476b8f1b08b69eeff76dfd2d119f816822"} err="failed to get container status \"f7f965b05c081819965360a985cc73476b8f1b08b69eeff76dfd2d119f816822\": rpc error: code = NotFound desc = could not find container \"f7f965b05c081819965360a985cc73476b8f1b08b69eeff76dfd2d119f816822\": container with ID starting with f7f965b05c081819965360a985cc73476b8f1b08b69eeff76dfd2d119f816822 not found: ID does not exist" Jul 07 11:13:44.702013 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.701981 2562 scope.go:117] "RemoveContainer" containerID="69ecb11f604d74c323cefb8de01623ef2a58f38b7b2ba3a9214143b2b494ec6d" Jul 07 11:13:44.702194 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:13:44.702179 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"69ecb11f604d74c323cefb8de01623ef2a58f38b7b2ba3a9214143b2b494ec6d\": container with ID starting with 69ecb11f604d74c323cefb8de01623ef2a58f38b7b2ba3a9214143b2b494ec6d not found: ID does not exist" containerID="69ecb11f604d74c323cefb8de01623ef2a58f38b7b2ba3a9214143b2b494ec6d" Jul 07 11:13:44.702235 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.702197 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"69ecb11f604d74c323cefb8de01623ef2a58f38b7b2ba3a9214143b2b494ec6d"} err="failed to get container status \"69ecb11f604d74c323cefb8de01623ef2a58f38b7b2ba3a9214143b2b494ec6d\": rpc error: code = NotFound desc = could not find container \"69ecb11f604d74c323cefb8de01623ef2a58f38b7b2ba3a9214143b2b494ec6d\": container with ID starting with 69ecb11f604d74c323cefb8de01623ef2a58f38b7b2ba3a9214143b2b494ec6d not found: ID does not exist" Jul 07 11:13:44.702235 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.702215 2562 scope.go:117] "RemoveContainer" containerID="e98438c43873adcbb713a7890ec5d37b1f1619423ae36cc69c9dc54490253c68" Jul 07 11:13:44.702400 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:13:44.702385 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e98438c43873adcbb713a7890ec5d37b1f1619423ae36cc69c9dc54490253c68\": container with ID starting with e98438c43873adcbb713a7890ec5d37b1f1619423ae36cc69c9dc54490253c68 not found: ID does not exist" containerID="e98438c43873adcbb713a7890ec5d37b1f1619423ae36cc69c9dc54490253c68" Jul 07 11:13:44.702438 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.702402 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e98438c43873adcbb713a7890ec5d37b1f1619423ae36cc69c9dc54490253c68"} err="failed to get container status \"e98438c43873adcbb713a7890ec5d37b1f1619423ae36cc69c9dc54490253c68\": rpc error: code = NotFound desc = could not find container \"e98438c43873adcbb713a7890ec5d37b1f1619423ae36cc69c9dc54490253c68\": container with ID starting with e98438c43873adcbb713a7890ec5d37b1f1619423ae36cc69c9dc54490253c68 not found: ID does not exist" Jul 07 11:13:44.702438 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.702413 2562 scope.go:117] "RemoveContainer" containerID="540adf46eb218874649adb7703958545843d4594f01f897d43762bc2cf395532" Jul 07 11:13:44.702628 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:13:44.702610 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"540adf46eb218874649adb7703958545843d4594f01f897d43762bc2cf395532\": container with ID starting with 540adf46eb218874649adb7703958545843d4594f01f897d43762bc2cf395532 not found: ID does not exist" containerID="540adf46eb218874649adb7703958545843d4594f01f897d43762bc2cf395532" Jul 07 11:13:44.702680 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.702632 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"540adf46eb218874649adb7703958545843d4594f01f897d43762bc2cf395532"} err="failed to get container status \"540adf46eb218874649adb7703958545843d4594f01f897d43762bc2cf395532\": rpc error: code = NotFound desc = could not find container \"540adf46eb218874649adb7703958545843d4594f01f897d43762bc2cf395532\": container with ID starting with 540adf46eb218874649adb7703958545843d4594f01f897d43762bc2cf395532 not found: ID does not exist" Jul 07 11:13:44.702680 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.702647 2562 scope.go:117] "RemoveContainer" containerID="39d4460ed91020e9f75de9cec8cd0334ef6c845ef86544427f906b3b7d1eb463" Jul 07 11:13:44.702873 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.702858 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"39d4460ed91020e9f75de9cec8cd0334ef6c845ef86544427f906b3b7d1eb463"} err="failed to get container status \"39d4460ed91020e9f75de9cec8cd0334ef6c845ef86544427f906b3b7d1eb463\": rpc error: code = NotFound desc = could not find container \"39d4460ed91020e9f75de9cec8cd0334ef6c845ef86544427f906b3b7d1eb463\": container with ID starting with 39d4460ed91020e9f75de9cec8cd0334ef6c845ef86544427f906b3b7d1eb463 not found: ID does not exist" Jul 07 11:13:44.702937 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.702873 2562 scope.go:117] "RemoveContainer" containerID="06bf9b5a2354ff69ec76d04926974872d40a7e36cedbb69c51792d0c84383954" Jul 07 11:13:44.703076 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.703062 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"06bf9b5a2354ff69ec76d04926974872d40a7e36cedbb69c51792d0c84383954"} err="failed to get container status \"06bf9b5a2354ff69ec76d04926974872d40a7e36cedbb69c51792d0c84383954\": rpc error: code = NotFound desc = could not find container \"06bf9b5a2354ff69ec76d04926974872d40a7e36cedbb69c51792d0c84383954\": container with ID starting with 06bf9b5a2354ff69ec76d04926974872d40a7e36cedbb69c51792d0c84383954 not found: ID does not exist" Jul 07 11:13:44.703117 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.703077 2562 scope.go:117] "RemoveContainer" containerID="9bddc02f54e220f34e3c00d4af2883a1576cd50ea4fcba3ed0450e695db4105d" Jul 07 11:13:44.703255 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.703241 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9bddc02f54e220f34e3c00d4af2883a1576cd50ea4fcba3ed0450e695db4105d"} err="failed to get container status \"9bddc02f54e220f34e3c00d4af2883a1576cd50ea4fcba3ed0450e695db4105d\": rpc error: code = NotFound desc = could not find container \"9bddc02f54e220f34e3c00d4af2883a1576cd50ea4fcba3ed0450e695db4105d\": container with ID starting with 9bddc02f54e220f34e3c00d4af2883a1576cd50ea4fcba3ed0450e695db4105d not found: ID does not exist" Jul 07 11:13:44.703296 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.703254 2562 scope.go:117] "RemoveContainer" containerID="f7f965b05c081819965360a985cc73476b8f1b08b69eeff76dfd2d119f816822" Jul 07 11:13:44.703463 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.703447 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f7f965b05c081819965360a985cc73476b8f1b08b69eeff76dfd2d119f816822"} err="failed to get container status \"f7f965b05c081819965360a985cc73476b8f1b08b69eeff76dfd2d119f816822\": rpc error: code = NotFound desc = could not find container \"f7f965b05c081819965360a985cc73476b8f1b08b69eeff76dfd2d119f816822\": container with ID starting with f7f965b05c081819965360a985cc73476b8f1b08b69eeff76dfd2d119f816822 not found: ID does not exist" Jul 07 11:13:44.703509 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.703463 2562 scope.go:117] "RemoveContainer" containerID="69ecb11f604d74c323cefb8de01623ef2a58f38b7b2ba3a9214143b2b494ec6d" Jul 07 11:13:44.703654 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.703639 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"69ecb11f604d74c323cefb8de01623ef2a58f38b7b2ba3a9214143b2b494ec6d"} err="failed to get container status \"69ecb11f604d74c323cefb8de01623ef2a58f38b7b2ba3a9214143b2b494ec6d\": rpc error: code = NotFound desc = could not find container \"69ecb11f604d74c323cefb8de01623ef2a58f38b7b2ba3a9214143b2b494ec6d\": container with ID starting with 69ecb11f604d74c323cefb8de01623ef2a58f38b7b2ba3a9214143b2b494ec6d not found: ID does not exist" Jul 07 11:13:44.703699 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.703654 2562 scope.go:117] "RemoveContainer" containerID="e98438c43873adcbb713a7890ec5d37b1f1619423ae36cc69c9dc54490253c68" Jul 07 11:13:44.703853 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.703834 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e98438c43873adcbb713a7890ec5d37b1f1619423ae36cc69c9dc54490253c68"} err="failed to get container status \"e98438c43873adcbb713a7890ec5d37b1f1619423ae36cc69c9dc54490253c68\": rpc error: code = NotFound desc = could not find container \"e98438c43873adcbb713a7890ec5d37b1f1619423ae36cc69c9dc54490253c68\": container with ID starting with e98438c43873adcbb713a7890ec5d37b1f1619423ae36cc69c9dc54490253c68 not found: ID does not exist" Jul 07 11:13:44.703936 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.703855 2562 scope.go:117] "RemoveContainer" containerID="540adf46eb218874649adb7703958545843d4594f01f897d43762bc2cf395532" Jul 07 11:13:44.704133 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.704116 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"540adf46eb218874649adb7703958545843d4594f01f897d43762bc2cf395532"} err="failed to get container status \"540adf46eb218874649adb7703958545843d4594f01f897d43762bc2cf395532\": rpc error: code = NotFound desc = could not find container \"540adf46eb218874649adb7703958545843d4594f01f897d43762bc2cf395532\": container with ID starting with 540adf46eb218874649adb7703958545843d4594f01f897d43762bc2cf395532 not found: ID does not exist" Jul 07 11:13:44.704181 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.704135 2562 scope.go:117] "RemoveContainer" containerID="39d4460ed91020e9f75de9cec8cd0334ef6c845ef86544427f906b3b7d1eb463" Jul 07 11:13:44.704355 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.704338 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"39d4460ed91020e9f75de9cec8cd0334ef6c845ef86544427f906b3b7d1eb463"} err="failed to get container status \"39d4460ed91020e9f75de9cec8cd0334ef6c845ef86544427f906b3b7d1eb463\": rpc error: code = NotFound desc = could not find container \"39d4460ed91020e9f75de9cec8cd0334ef6c845ef86544427f906b3b7d1eb463\": container with ID starting with 39d4460ed91020e9f75de9cec8cd0334ef6c845ef86544427f906b3b7d1eb463 not found: ID does not exist" Jul 07 11:13:44.704420 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.704357 2562 scope.go:117] "RemoveContainer" containerID="06bf9b5a2354ff69ec76d04926974872d40a7e36cedbb69c51792d0c84383954" Jul 07 11:13:44.704581 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.704564 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"06bf9b5a2354ff69ec76d04926974872d40a7e36cedbb69c51792d0c84383954"} err="failed to get container status \"06bf9b5a2354ff69ec76d04926974872d40a7e36cedbb69c51792d0c84383954\": rpc error: code = NotFound desc = could not find container \"06bf9b5a2354ff69ec76d04926974872d40a7e36cedbb69c51792d0c84383954\": container with ID starting with 06bf9b5a2354ff69ec76d04926974872d40a7e36cedbb69c51792d0c84383954 not found: ID does not exist" Jul 07 11:13:44.704629 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.704581 2562 scope.go:117] "RemoveContainer" containerID="9bddc02f54e220f34e3c00d4af2883a1576cd50ea4fcba3ed0450e695db4105d" Jul 07 11:13:44.704777 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.704759 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9bddc02f54e220f34e3c00d4af2883a1576cd50ea4fcba3ed0450e695db4105d"} err="failed to get container status \"9bddc02f54e220f34e3c00d4af2883a1576cd50ea4fcba3ed0450e695db4105d\": rpc error: code = NotFound desc = could not find container \"9bddc02f54e220f34e3c00d4af2883a1576cd50ea4fcba3ed0450e695db4105d\": container with ID starting with 9bddc02f54e220f34e3c00d4af2883a1576cd50ea4fcba3ed0450e695db4105d not found: ID does not exist" Jul 07 11:13:44.704818 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.704778 2562 scope.go:117] "RemoveContainer" containerID="f7f965b05c081819965360a985cc73476b8f1b08b69eeff76dfd2d119f816822" Jul 07 11:13:44.704999 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.704979 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f7f965b05c081819965360a985cc73476b8f1b08b69eeff76dfd2d119f816822"} err="failed to get container status \"f7f965b05c081819965360a985cc73476b8f1b08b69eeff76dfd2d119f816822\": rpc error: code = NotFound desc = could not find container \"f7f965b05c081819965360a985cc73476b8f1b08b69eeff76dfd2d119f816822\": container with ID starting with f7f965b05c081819965360a985cc73476b8f1b08b69eeff76dfd2d119f816822 not found: ID does not exist" Jul 07 11:13:44.704999 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.704995 2562 scope.go:117] "RemoveContainer" containerID="69ecb11f604d74c323cefb8de01623ef2a58f38b7b2ba3a9214143b2b494ec6d" Jul 07 11:13:44.705217 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.705200 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"69ecb11f604d74c323cefb8de01623ef2a58f38b7b2ba3a9214143b2b494ec6d"} err="failed to get container status \"69ecb11f604d74c323cefb8de01623ef2a58f38b7b2ba3a9214143b2b494ec6d\": rpc error: code = NotFound desc = could not find container \"69ecb11f604d74c323cefb8de01623ef2a58f38b7b2ba3a9214143b2b494ec6d\": container with ID starting with 69ecb11f604d74c323cefb8de01623ef2a58f38b7b2ba3a9214143b2b494ec6d not found: ID does not exist" Jul 07 11:13:44.705256 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.705217 2562 scope.go:117] "RemoveContainer" containerID="e98438c43873adcbb713a7890ec5d37b1f1619423ae36cc69c9dc54490253c68" Jul 07 11:13:44.705444 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.705426 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e98438c43873adcbb713a7890ec5d37b1f1619423ae36cc69c9dc54490253c68"} err="failed to get container status \"e98438c43873adcbb713a7890ec5d37b1f1619423ae36cc69c9dc54490253c68\": rpc error: code = NotFound desc = could not find container \"e98438c43873adcbb713a7890ec5d37b1f1619423ae36cc69c9dc54490253c68\": container with ID starting with e98438c43873adcbb713a7890ec5d37b1f1619423ae36cc69c9dc54490253c68 not found: ID does not exist" Jul 07 11:13:44.705444 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.705443 2562 scope.go:117] "RemoveContainer" containerID="540adf46eb218874649adb7703958545843d4594f01f897d43762bc2cf395532" Jul 07 11:13:44.705668 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.705650 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"540adf46eb218874649adb7703958545843d4594f01f897d43762bc2cf395532"} err="failed to get container status \"540adf46eb218874649adb7703958545843d4594f01f897d43762bc2cf395532\": rpc error: code = NotFound desc = could not find container \"540adf46eb218874649adb7703958545843d4594f01f897d43762bc2cf395532\": container with ID starting with 540adf46eb218874649adb7703958545843d4594f01f897d43762bc2cf395532 not found: ID does not exist" Jul 07 11:13:44.705706 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.705668 2562 scope.go:117] "RemoveContainer" containerID="39d4460ed91020e9f75de9cec8cd0334ef6c845ef86544427f906b3b7d1eb463" Jul 07 11:13:44.705869 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.705853 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"39d4460ed91020e9f75de9cec8cd0334ef6c845ef86544427f906b3b7d1eb463"} err="failed to get container status \"39d4460ed91020e9f75de9cec8cd0334ef6c845ef86544427f906b3b7d1eb463\": rpc error: code = NotFound desc = could not find container \"39d4460ed91020e9f75de9cec8cd0334ef6c845ef86544427f906b3b7d1eb463\": container with ID starting with 39d4460ed91020e9f75de9cec8cd0334ef6c845ef86544427f906b3b7d1eb463 not found: ID does not exist" Jul 07 11:13:44.705926 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.705869 2562 scope.go:117] "RemoveContainer" containerID="06bf9b5a2354ff69ec76d04926974872d40a7e36cedbb69c51792d0c84383954" Jul 07 11:13:44.706106 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.706090 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"06bf9b5a2354ff69ec76d04926974872d40a7e36cedbb69c51792d0c84383954"} err="failed to get container status \"06bf9b5a2354ff69ec76d04926974872d40a7e36cedbb69c51792d0c84383954\": rpc error: code = NotFound desc = could not find container \"06bf9b5a2354ff69ec76d04926974872d40a7e36cedbb69c51792d0c84383954\": container with ID starting with 06bf9b5a2354ff69ec76d04926974872d40a7e36cedbb69c51792d0c84383954 not found: ID does not exist" Jul 07 11:13:44.706106 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.706106 2562 scope.go:117] "RemoveContainer" containerID="9bddc02f54e220f34e3c00d4af2883a1576cd50ea4fcba3ed0450e695db4105d" Jul 07 11:13:44.706320 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.706300 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9bddc02f54e220f34e3c00d4af2883a1576cd50ea4fcba3ed0450e695db4105d"} err="failed to get container status \"9bddc02f54e220f34e3c00d4af2883a1576cd50ea4fcba3ed0450e695db4105d\": rpc error: code = NotFound desc = could not find container \"9bddc02f54e220f34e3c00d4af2883a1576cd50ea4fcba3ed0450e695db4105d\": container with ID starting with 9bddc02f54e220f34e3c00d4af2883a1576cd50ea4fcba3ed0450e695db4105d not found: ID does not exist" Jul 07 11:13:44.706320 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.706317 2562 scope.go:117] "RemoveContainer" containerID="f7f965b05c081819965360a985cc73476b8f1b08b69eeff76dfd2d119f816822" Jul 07 11:13:44.706587 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.706570 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f7f965b05c081819965360a985cc73476b8f1b08b69eeff76dfd2d119f816822"} err="failed to get container status \"f7f965b05c081819965360a985cc73476b8f1b08b69eeff76dfd2d119f816822\": rpc error: code = NotFound desc = could not find container \"f7f965b05c081819965360a985cc73476b8f1b08b69eeff76dfd2d119f816822\": container with ID starting with f7f965b05c081819965360a985cc73476b8f1b08b69eeff76dfd2d119f816822 not found: ID does not exist" Jul 07 11:13:44.706587 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.706588 2562 scope.go:117] "RemoveContainer" containerID="69ecb11f604d74c323cefb8de01623ef2a58f38b7b2ba3a9214143b2b494ec6d" Jul 07 11:13:44.706832 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.706809 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"69ecb11f604d74c323cefb8de01623ef2a58f38b7b2ba3a9214143b2b494ec6d"} err="failed to get container status \"69ecb11f604d74c323cefb8de01623ef2a58f38b7b2ba3a9214143b2b494ec6d\": rpc error: code = NotFound desc = could not find container \"69ecb11f604d74c323cefb8de01623ef2a58f38b7b2ba3a9214143b2b494ec6d\": container with ID starting with 69ecb11f604d74c323cefb8de01623ef2a58f38b7b2ba3a9214143b2b494ec6d not found: ID does not exist" Jul 07 11:13:44.706941 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.706834 2562 scope.go:117] "RemoveContainer" containerID="e98438c43873adcbb713a7890ec5d37b1f1619423ae36cc69c9dc54490253c68" Jul 07 11:13:44.707082 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.707065 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e98438c43873adcbb713a7890ec5d37b1f1619423ae36cc69c9dc54490253c68"} err="failed to get container status \"e98438c43873adcbb713a7890ec5d37b1f1619423ae36cc69c9dc54490253c68\": rpc error: code = NotFound desc = could not find container \"e98438c43873adcbb713a7890ec5d37b1f1619423ae36cc69c9dc54490253c68\": container with ID starting with e98438c43873adcbb713a7890ec5d37b1f1619423ae36cc69c9dc54490253c68 not found: ID does not exist" Jul 07 11:13:44.707139 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.707082 2562 scope.go:117] "RemoveContainer" containerID="540adf46eb218874649adb7703958545843d4594f01f897d43762bc2cf395532" Jul 07 11:13:44.707306 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.707291 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"540adf46eb218874649adb7703958545843d4594f01f897d43762bc2cf395532"} err="failed to get container status \"540adf46eb218874649adb7703958545843d4594f01f897d43762bc2cf395532\": rpc error: code = NotFound desc = could not find container \"540adf46eb218874649adb7703958545843d4594f01f897d43762bc2cf395532\": container with ID starting with 540adf46eb218874649adb7703958545843d4594f01f897d43762bc2cf395532 not found: ID does not exist" Jul 07 11:13:44.707306 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.707306 2562 scope.go:117] "RemoveContainer" containerID="39d4460ed91020e9f75de9cec8cd0334ef6c845ef86544427f906b3b7d1eb463" Jul 07 11:13:44.707526 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.707503 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"39d4460ed91020e9f75de9cec8cd0334ef6c845ef86544427f906b3b7d1eb463"} err="failed to get container status \"39d4460ed91020e9f75de9cec8cd0334ef6c845ef86544427f906b3b7d1eb463\": rpc error: code = NotFound desc = could not find container \"39d4460ed91020e9f75de9cec8cd0334ef6c845ef86544427f906b3b7d1eb463\": container with ID starting with 39d4460ed91020e9f75de9cec8cd0334ef6c845ef86544427f906b3b7d1eb463 not found: ID does not exist" Jul 07 11:13:44.707526 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.707519 2562 scope.go:117] "RemoveContainer" containerID="06bf9b5a2354ff69ec76d04926974872d40a7e36cedbb69c51792d0c84383954" Jul 07 11:13:44.707752 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.707736 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"06bf9b5a2354ff69ec76d04926974872d40a7e36cedbb69c51792d0c84383954"} err="failed to get container status \"06bf9b5a2354ff69ec76d04926974872d40a7e36cedbb69c51792d0c84383954\": rpc error: code = NotFound desc = could not find container \"06bf9b5a2354ff69ec76d04926974872d40a7e36cedbb69c51792d0c84383954\": container with ID starting with 06bf9b5a2354ff69ec76d04926974872d40a7e36cedbb69c51792d0c84383954 not found: ID does not exist" Jul 07 11:13:44.707795 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.707753 2562 scope.go:117] "RemoveContainer" containerID="9bddc02f54e220f34e3c00d4af2883a1576cd50ea4fcba3ed0450e695db4105d" Jul 07 11:13:44.707959 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.707940 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9bddc02f54e220f34e3c00d4af2883a1576cd50ea4fcba3ed0450e695db4105d"} err="failed to get container status \"9bddc02f54e220f34e3c00d4af2883a1576cd50ea4fcba3ed0450e695db4105d\": rpc error: code = NotFound desc = could not find container \"9bddc02f54e220f34e3c00d4af2883a1576cd50ea4fcba3ed0450e695db4105d\": container with ID starting with 9bddc02f54e220f34e3c00d4af2883a1576cd50ea4fcba3ed0450e695db4105d not found: ID does not exist" Jul 07 11:13:44.708007 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.707960 2562 scope.go:117] "RemoveContainer" containerID="f7f965b05c081819965360a985cc73476b8f1b08b69eeff76dfd2d119f816822" Jul 07 11:13:44.708206 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.708185 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f7f965b05c081819965360a985cc73476b8f1b08b69eeff76dfd2d119f816822"} err="failed to get container status \"f7f965b05c081819965360a985cc73476b8f1b08b69eeff76dfd2d119f816822\": rpc error: code = NotFound desc = could not find container \"f7f965b05c081819965360a985cc73476b8f1b08b69eeff76dfd2d119f816822\": container with ID starting with f7f965b05c081819965360a985cc73476b8f1b08b69eeff76dfd2d119f816822 not found: ID does not exist" Jul 07 11:13:44.708206 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.708199 2562 scope.go:117] "RemoveContainer" containerID="69ecb11f604d74c323cefb8de01623ef2a58f38b7b2ba3a9214143b2b494ec6d" Jul 07 11:13:44.708471 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.708450 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"69ecb11f604d74c323cefb8de01623ef2a58f38b7b2ba3a9214143b2b494ec6d"} err="failed to get container status \"69ecb11f604d74c323cefb8de01623ef2a58f38b7b2ba3a9214143b2b494ec6d\": rpc error: code = NotFound desc = could not find container \"69ecb11f604d74c323cefb8de01623ef2a58f38b7b2ba3a9214143b2b494ec6d\": container with ID starting with 69ecb11f604d74c323cefb8de01623ef2a58f38b7b2ba3a9214143b2b494ec6d not found: ID does not exist" Jul 07 11:13:44.708471 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.708471 2562 scope.go:117] "RemoveContainer" containerID="e98438c43873adcbb713a7890ec5d37b1f1619423ae36cc69c9dc54490253c68" Jul 07 11:13:44.708708 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.708689 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e98438c43873adcbb713a7890ec5d37b1f1619423ae36cc69c9dc54490253c68"} err="failed to get container status \"e98438c43873adcbb713a7890ec5d37b1f1619423ae36cc69c9dc54490253c68\": rpc error: code = NotFound desc = could not find container \"e98438c43873adcbb713a7890ec5d37b1f1619423ae36cc69c9dc54490253c68\": container with ID starting with e98438c43873adcbb713a7890ec5d37b1f1619423ae36cc69c9dc54490253c68 not found: ID does not exist" Jul 07 11:13:44.708708 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.708708 2562 scope.go:117] "RemoveContainer" containerID="540adf46eb218874649adb7703958545843d4594f01f897d43762bc2cf395532" Jul 07 11:13:44.709015 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.708998 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"540adf46eb218874649adb7703958545843d4594f01f897d43762bc2cf395532"} err="failed to get container status \"540adf46eb218874649adb7703958545843d4594f01f897d43762bc2cf395532\": rpc error: code = NotFound desc = could not find container \"540adf46eb218874649adb7703958545843d4594f01f897d43762bc2cf395532\": container with ID starting with 540adf46eb218874649adb7703958545843d4594f01f897d43762bc2cf395532 not found: ID does not exist" Jul 07 11:13:44.709074 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.709015 2562 scope.go:117] "RemoveContainer" containerID="39d4460ed91020e9f75de9cec8cd0334ef6c845ef86544427f906b3b7d1eb463" Jul 07 11:13:44.709255 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.709238 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"39d4460ed91020e9f75de9cec8cd0334ef6c845ef86544427f906b3b7d1eb463"} err="failed to get container status \"39d4460ed91020e9f75de9cec8cd0334ef6c845ef86544427f906b3b7d1eb463\": rpc error: code = NotFound desc = could not find container \"39d4460ed91020e9f75de9cec8cd0334ef6c845ef86544427f906b3b7d1eb463\": container with ID starting with 39d4460ed91020e9f75de9cec8cd0334ef6c845ef86544427f906b3b7d1eb463 not found: ID does not exist" Jul 07 11:13:44.709297 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.709257 2562 scope.go:117] "RemoveContainer" containerID="06bf9b5a2354ff69ec76d04926974872d40a7e36cedbb69c51792d0c84383954" Jul 07 11:13:44.709482 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.709465 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"06bf9b5a2354ff69ec76d04926974872d40a7e36cedbb69c51792d0c84383954"} err="failed to get container status \"06bf9b5a2354ff69ec76d04926974872d40a7e36cedbb69c51792d0c84383954\": rpc error: code = NotFound desc = could not find container \"06bf9b5a2354ff69ec76d04926974872d40a7e36cedbb69c51792d0c84383954\": container with ID starting with 06bf9b5a2354ff69ec76d04926974872d40a7e36cedbb69c51792d0c84383954 not found: ID does not exist" Jul 07 11:13:44.709482 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.709481 2562 scope.go:117] "RemoveContainer" containerID="9bddc02f54e220f34e3c00d4af2883a1576cd50ea4fcba3ed0450e695db4105d" Jul 07 11:13:44.709704 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.709688 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9bddc02f54e220f34e3c00d4af2883a1576cd50ea4fcba3ed0450e695db4105d"} err="failed to get container status \"9bddc02f54e220f34e3c00d4af2883a1576cd50ea4fcba3ed0450e695db4105d\": rpc error: code = NotFound desc = could not find container \"9bddc02f54e220f34e3c00d4af2883a1576cd50ea4fcba3ed0450e695db4105d\": container with ID starting with 9bddc02f54e220f34e3c00d4af2883a1576cd50ea4fcba3ed0450e695db4105d not found: ID does not exist" Jul 07 11:13:44.709757 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.709705 2562 scope.go:117] "RemoveContainer" containerID="f7f965b05c081819965360a985cc73476b8f1b08b69eeff76dfd2d119f816822" Jul 07 11:13:44.709905 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.709889 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f7f965b05c081819965360a985cc73476b8f1b08b69eeff76dfd2d119f816822"} err="failed to get container status \"f7f965b05c081819965360a985cc73476b8f1b08b69eeff76dfd2d119f816822\": rpc error: code = NotFound desc = could not find container \"f7f965b05c081819965360a985cc73476b8f1b08b69eeff76dfd2d119f816822\": container with ID starting with f7f965b05c081819965360a985cc73476b8f1b08b69eeff76dfd2d119f816822 not found: ID does not exist" Jul 07 11:13:44.709977 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.709907 2562 scope.go:117] "RemoveContainer" containerID="69ecb11f604d74c323cefb8de01623ef2a58f38b7b2ba3a9214143b2b494ec6d" Jul 07 11:13:44.710148 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.710131 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"69ecb11f604d74c323cefb8de01623ef2a58f38b7b2ba3a9214143b2b494ec6d"} err="failed to get container status \"69ecb11f604d74c323cefb8de01623ef2a58f38b7b2ba3a9214143b2b494ec6d\": rpc error: code = NotFound desc = could not find container \"69ecb11f604d74c323cefb8de01623ef2a58f38b7b2ba3a9214143b2b494ec6d\": container with ID starting with 69ecb11f604d74c323cefb8de01623ef2a58f38b7b2ba3a9214143b2b494ec6d not found: ID does not exist" Jul 07 11:13:44.710201 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.710149 2562 scope.go:117] "RemoveContainer" containerID="e98438c43873adcbb713a7890ec5d37b1f1619423ae36cc69c9dc54490253c68" Jul 07 11:13:44.710359 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.710342 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e98438c43873adcbb713a7890ec5d37b1f1619423ae36cc69c9dc54490253c68"} err="failed to get container status \"e98438c43873adcbb713a7890ec5d37b1f1619423ae36cc69c9dc54490253c68\": rpc error: code = NotFound desc = could not find container \"e98438c43873adcbb713a7890ec5d37b1f1619423ae36cc69c9dc54490253c68\": container with ID starting with e98438c43873adcbb713a7890ec5d37b1f1619423ae36cc69c9dc54490253c68 not found: ID does not exist" Jul 07 11:13:44.710409 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.710361 2562 scope.go:117] "RemoveContainer" containerID="540adf46eb218874649adb7703958545843d4594f01f897d43762bc2cf395532" Jul 07 11:13:44.710535 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.710521 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"540adf46eb218874649adb7703958545843d4594f01f897d43762bc2cf395532"} err="failed to get container status \"540adf46eb218874649adb7703958545843d4594f01f897d43762bc2cf395532\": rpc error: code = NotFound desc = could not find container \"540adf46eb218874649adb7703958545843d4594f01f897d43762bc2cf395532\": container with ID starting with 540adf46eb218874649adb7703958545843d4594f01f897d43762bc2cf395532 not found: ID does not exist" Jul 07 11:13:44.720356 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.720340 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 07 11:13:44.720628 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.720617 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c088023f-86df-4531-be68-1a21bad47ae6" containerName="init-config-reloader" Jul 07 11:13:44.720668 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.720630 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="c088023f-86df-4531-be68-1a21bad47ae6" containerName="init-config-reloader" Jul 07 11:13:44.720668 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.720646 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c088023f-86df-4531-be68-1a21bad47ae6" containerName="config-reloader" Jul 07 11:13:44.720668 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.720653 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="c088023f-86df-4531-be68-1a21bad47ae6" containerName="config-reloader" Jul 07 11:13:44.720668 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.720658 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c088023f-86df-4531-be68-1a21bad47ae6" containerName="kube-rbac-proxy" Jul 07 11:13:44.720668 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.720663 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="c088023f-86df-4531-be68-1a21bad47ae6" containerName="kube-rbac-proxy" Jul 07 11:13:44.720811 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.720672 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c088023f-86df-4531-be68-1a21bad47ae6" containerName="kube-rbac-proxy-thanos" Jul 07 11:13:44.720811 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.720677 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="c088023f-86df-4531-be68-1a21bad47ae6" containerName="kube-rbac-proxy-thanos" Jul 07 11:13:44.720811 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.720688 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c088023f-86df-4531-be68-1a21bad47ae6" containerName="thanos-sidecar" Jul 07 11:13:44.720811 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.720694 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="c088023f-86df-4531-be68-1a21bad47ae6" containerName="thanos-sidecar" Jul 07 11:13:44.720811 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.720701 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c088023f-86df-4531-be68-1a21bad47ae6" containerName="kube-rbac-proxy-web" Jul 07 11:13:44.720811 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.720706 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="c088023f-86df-4531-be68-1a21bad47ae6" containerName="kube-rbac-proxy-web" Jul 07 11:13:44.720811 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.720712 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c088023f-86df-4531-be68-1a21bad47ae6" containerName="prometheus" Jul 07 11:13:44.720811 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.720717 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="c088023f-86df-4531-be68-1a21bad47ae6" containerName="prometheus" Jul 07 11:13:44.720811 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.720761 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="c088023f-86df-4531-be68-1a21bad47ae6" containerName="thanos-sidecar" Jul 07 11:13:44.720811 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.720769 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="c088023f-86df-4531-be68-1a21bad47ae6" containerName="config-reloader" Jul 07 11:13:44.720811 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.720775 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="c088023f-86df-4531-be68-1a21bad47ae6" containerName="kube-rbac-proxy-web" Jul 07 11:13:44.720811 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.720780 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="c088023f-86df-4531-be68-1a21bad47ae6" containerName="kube-rbac-proxy-thanos" Jul 07 11:13:44.720811 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.720786 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="c088023f-86df-4531-be68-1a21bad47ae6" containerName="kube-rbac-proxy" Jul 07 11:13:44.720811 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.720792 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="c088023f-86df-4531-be68-1a21bad47ae6" containerName="prometheus" Jul 07 11:13:44.724559 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.724528 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:13:44.728359 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.728330 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 07 11:13:44.728580 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.728561 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 07 11:13:44.728656 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.728603 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 07 11:13:44.728908 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.728885 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-nln8s\"" Jul 07 11:13:44.729518 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.729501 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 07 11:13:44.729588 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.729501 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 07 11:13:44.729881 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.729723 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 07 11:13:44.729881 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.729761 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 07 11:13:44.729881 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.729787 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 07 11:13:44.729881 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.729815 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 07 11:13:44.729881 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.729885 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 07 11:13:44.730178 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.730046 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 07 11:13:44.730178 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.730135 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-9jb136q3vmpab\"" Jul 07 11:13:44.730808 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.730792 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 07 11:13:44.739107 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.739085 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 07 11:13:44.741363 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.741348 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 07 11:13:44.745889 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.745850 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 07 11:13:44.748969 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.748948 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:13:44.778490 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.778461 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/67e0c033-c10d-4952-883c-56e13a8f2e23-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"67e0c033-c10d-4952-883c-56e13a8f2e23\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:13:44.778590 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.778503 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/67e0c033-c10d-4952-883c-56e13a8f2e23-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"67e0c033-c10d-4952-883c-56e13a8f2e23\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:13:44.778590 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.778536 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/67e0c033-c10d-4952-883c-56e13a8f2e23-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"67e0c033-c10d-4952-883c-56e13a8f2e23\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:13:44.778590 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.778565 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/67e0c033-c10d-4952-883c-56e13a8f2e23-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"67e0c033-c10d-4952-883c-56e13a8f2e23\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:13:44.778716 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.778616 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/67e0c033-c10d-4952-883c-56e13a8f2e23-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"67e0c033-c10d-4952-883c-56e13a8f2e23\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:13:44.778716 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.778698 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wk6m8\" (UniqueName: \"kubernetes.io/projected/67e0c033-c10d-4952-883c-56e13a8f2e23-kube-api-access-wk6m8\") pod \"prometheus-k8s-0\" (UID: \"67e0c033-c10d-4952-883c-56e13a8f2e23\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:13:44.778793 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.778730 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/67e0c033-c10d-4952-883c-56e13a8f2e23-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"67e0c033-c10d-4952-883c-56e13a8f2e23\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:13:44.778793 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.778773 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/67e0c033-c10d-4952-883c-56e13a8f2e23-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"67e0c033-c10d-4952-883c-56e13a8f2e23\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:13:44.778878 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.778813 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/67e0c033-c10d-4952-883c-56e13a8f2e23-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"67e0c033-c10d-4952-883c-56e13a8f2e23\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:13:44.778878 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.778840 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/67e0c033-c10d-4952-883c-56e13a8f2e23-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"67e0c033-c10d-4952-883c-56e13a8f2e23\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:13:44.778997 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.778881 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/67e0c033-c10d-4952-883c-56e13a8f2e23-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"67e0c033-c10d-4952-883c-56e13a8f2e23\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:13:44.778997 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.778980 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/67e0c033-c10d-4952-883c-56e13a8f2e23-config\") pod \"prometheus-k8s-0\" (UID: \"67e0c033-c10d-4952-883c-56e13a8f2e23\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:13:44.779141 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.779038 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/67e0c033-c10d-4952-883c-56e13a8f2e23-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"67e0c033-c10d-4952-883c-56e13a8f2e23\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:13:44.779141 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.779063 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/67e0c033-c10d-4952-883c-56e13a8f2e23-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"67e0c033-c10d-4952-883c-56e13a8f2e23\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:13:44.779141 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.779102 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/67e0c033-c10d-4952-883c-56e13a8f2e23-web-config\") pod \"prometheus-k8s-0\" (UID: \"67e0c033-c10d-4952-883c-56e13a8f2e23\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:13:44.779141 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.779125 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/67e0c033-c10d-4952-883c-56e13a8f2e23-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"67e0c033-c10d-4952-883c-56e13a8f2e23\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:13:44.779350 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.779145 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/67e0c033-c10d-4952-883c-56e13a8f2e23-config-out\") pod \"prometheus-k8s-0\" (UID: \"67e0c033-c10d-4952-883c-56e13a8f2e23\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:13:44.779350 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.779179 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/67e0c033-c10d-4952-883c-56e13a8f2e23-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"67e0c033-c10d-4952-883c-56e13a8f2e23\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:13:44.779350 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.779233 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/67e0c033-c10d-4952-883c-56e13a8f2e23-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"67e0c033-c10d-4952-883c-56e13a8f2e23\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:13:44.779350 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.779283 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/67e0c033-c10d-4952-883c-56e13a8f2e23-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"67e0c033-c10d-4952-883c-56e13a8f2e23\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:13:44.880364 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.880345 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wk6m8\" (UniqueName: \"kubernetes.io/projected/67e0c033-c10d-4952-883c-56e13a8f2e23-kube-api-access-wk6m8\") pod \"prometheus-k8s-0\" (UID: \"67e0c033-c10d-4952-883c-56e13a8f2e23\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:13:44.880747 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.880378 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/67e0c033-c10d-4952-883c-56e13a8f2e23-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"67e0c033-c10d-4952-883c-56e13a8f2e23\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:13:44.880747 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.880399 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/67e0c033-c10d-4952-883c-56e13a8f2e23-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"67e0c033-c10d-4952-883c-56e13a8f2e23\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:13:44.880747 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.880427 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/67e0c033-c10d-4952-883c-56e13a8f2e23-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"67e0c033-c10d-4952-883c-56e13a8f2e23\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:13:44.880747 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.880444 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/67e0c033-c10d-4952-883c-56e13a8f2e23-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"67e0c033-c10d-4952-883c-56e13a8f2e23\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:13:44.880747 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.880482 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/67e0c033-c10d-4952-883c-56e13a8f2e23-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"67e0c033-c10d-4952-883c-56e13a8f2e23\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:13:44.880747 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.880508 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/67e0c033-c10d-4952-883c-56e13a8f2e23-config\") pod \"prometheus-k8s-0\" (UID: \"67e0c033-c10d-4952-883c-56e13a8f2e23\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:13:44.880747 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.880533 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/67e0c033-c10d-4952-883c-56e13a8f2e23-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"67e0c033-c10d-4952-883c-56e13a8f2e23\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:13:44.880747 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.880567 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/67e0c033-c10d-4952-883c-56e13a8f2e23-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"67e0c033-c10d-4952-883c-56e13a8f2e23\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:13:44.880747 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.880632 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/67e0c033-c10d-4952-883c-56e13a8f2e23-web-config\") pod \"prometheus-k8s-0\" (UID: \"67e0c033-c10d-4952-883c-56e13a8f2e23\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:13:44.880747 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.880657 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/67e0c033-c10d-4952-883c-56e13a8f2e23-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"67e0c033-c10d-4952-883c-56e13a8f2e23\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:13:44.880747 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.880681 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/67e0c033-c10d-4952-883c-56e13a8f2e23-config-out\") pod \"prometheus-k8s-0\" (UID: \"67e0c033-c10d-4952-883c-56e13a8f2e23\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:13:44.880747 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.880737 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/67e0c033-c10d-4952-883c-56e13a8f2e23-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"67e0c033-c10d-4952-883c-56e13a8f2e23\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:13:44.881395 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.880764 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/67e0c033-c10d-4952-883c-56e13a8f2e23-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"67e0c033-c10d-4952-883c-56e13a8f2e23\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:13:44.881395 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.880787 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/67e0c033-c10d-4952-883c-56e13a8f2e23-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"67e0c033-c10d-4952-883c-56e13a8f2e23\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:13:44.881395 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.880810 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/67e0c033-c10d-4952-883c-56e13a8f2e23-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"67e0c033-c10d-4952-883c-56e13a8f2e23\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:13:44.881395 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.880876 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/67e0c033-c10d-4952-883c-56e13a8f2e23-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"67e0c033-c10d-4952-883c-56e13a8f2e23\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:13:44.881395 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.880948 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/67e0c033-c10d-4952-883c-56e13a8f2e23-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"67e0c033-c10d-4952-883c-56e13a8f2e23\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:13:44.881395 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.880982 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/67e0c033-c10d-4952-883c-56e13a8f2e23-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"67e0c033-c10d-4952-883c-56e13a8f2e23\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:13:44.881395 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.881007 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/67e0c033-c10d-4952-883c-56e13a8f2e23-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"67e0c033-c10d-4952-883c-56e13a8f2e23\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:13:44.881395 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.881239 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/67e0c033-c10d-4952-883c-56e13a8f2e23-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"67e0c033-c10d-4952-883c-56e13a8f2e23\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:13:44.881395 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.881386 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/67e0c033-c10d-4952-883c-56e13a8f2e23-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"67e0c033-c10d-4952-883c-56e13a8f2e23\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:13:44.881860 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.881413 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/67e0c033-c10d-4952-883c-56e13a8f2e23-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"67e0c033-c10d-4952-883c-56e13a8f2e23\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:13:44.881860 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.881679 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/67e0c033-c10d-4952-883c-56e13a8f2e23-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"67e0c033-c10d-4952-883c-56e13a8f2e23\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:13:44.882658 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.882635 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/67e0c033-c10d-4952-883c-56e13a8f2e23-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"67e0c033-c10d-4952-883c-56e13a8f2e23\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:13:44.884333 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.883503 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/67e0c033-c10d-4952-883c-56e13a8f2e23-config\") pod \"prometheus-k8s-0\" (UID: \"67e0c033-c10d-4952-883c-56e13a8f2e23\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:13:44.884333 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.883623 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/67e0c033-c10d-4952-883c-56e13a8f2e23-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"67e0c033-c10d-4952-883c-56e13a8f2e23\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:13:44.884333 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.883991 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/67e0c033-c10d-4952-883c-56e13a8f2e23-web-config\") pod \"prometheus-k8s-0\" (UID: \"67e0c033-c10d-4952-883c-56e13a8f2e23\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:13:44.884333 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.884126 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/67e0c033-c10d-4952-883c-56e13a8f2e23-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"67e0c033-c10d-4952-883c-56e13a8f2e23\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:13:44.884333 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.884325 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/67e0c033-c10d-4952-883c-56e13a8f2e23-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"67e0c033-c10d-4952-883c-56e13a8f2e23\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:13:44.884702 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.884467 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/67e0c033-c10d-4952-883c-56e13a8f2e23-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"67e0c033-c10d-4952-883c-56e13a8f2e23\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:13:44.884809 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.884785 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/67e0c033-c10d-4952-883c-56e13a8f2e23-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"67e0c033-c10d-4952-883c-56e13a8f2e23\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:13:44.884901 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.884822 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/67e0c033-c10d-4952-883c-56e13a8f2e23-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"67e0c033-c10d-4952-883c-56e13a8f2e23\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:13:44.885088 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.885064 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/67e0c033-c10d-4952-883c-56e13a8f2e23-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"67e0c033-c10d-4952-883c-56e13a8f2e23\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:13:44.885527 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.885488 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:13:44.886506 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.886476 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/67e0c033-c10d-4952-883c-56e13a8f2e23-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"67e0c033-c10d-4952-883c-56e13a8f2e23\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:13:44.886845 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.886822 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/67e0c033-c10d-4952-883c-56e13a8f2e23-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"67e0c033-c10d-4952-883c-56e13a8f2e23\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:13:44.886945 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.886870 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/67e0c033-c10d-4952-883c-56e13a8f2e23-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"67e0c033-c10d-4952-883c-56e13a8f2e23\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:13:44.887056 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.887041 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/67e0c033-c10d-4952-883c-56e13a8f2e23-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"67e0c033-c10d-4952-883c-56e13a8f2e23\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:13:44.887404 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.887389 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/67e0c033-c10d-4952-883c-56e13a8f2e23-config-out\") pod \"prometheus-k8s-0\" (UID: \"67e0c033-c10d-4952-883c-56e13a8f2e23\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:13:44.889460 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:44.889437 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wk6m8\" (UniqueName: \"kubernetes.io/projected/67e0c033-c10d-4952-883c-56e13a8f2e23-kube-api-access-wk6m8\") pod \"prometheus-k8s-0\" (UID: \"67e0c033-c10d-4952-883c-56e13a8f2e23\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:13:45.034396 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:45.034377 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:13:45.175758 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:45.175730 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 07 11:13:45.177598 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:13:45.177575 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67e0c033_c10d_4952_883c_56e13a8f2e23.slice/crio-629362cd5530dcd19e907aa1a51cc921bb23fc821cbbc1f2a583f1f53289b1d6 WatchSource:0}: Error finding container 629362cd5530dcd19e907aa1a51cc921bb23fc821cbbc1f2a583f1f53289b1d6: Status 404 returned error can't find the container with id 629362cd5530dcd19e907aa1a51cc921bb23fc821cbbc1f2a583f1f53289b1d6 Jul 07 11:13:45.399411 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:45.399383 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:13:45.658275 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:45.658244 2562 generic.go:356] "Generic (PLEG): container finished" podID="67e0c033-c10d-4952-883c-56e13a8f2e23" containerID="707d7b64202369487ef2689f2d01880e28058e64db38d0f3d6d3159890e685ad" exitCode=0 Jul 07 11:13:45.658405 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:45.658292 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"67e0c033-c10d-4952-883c-56e13a8f2e23","Type":"ContainerDied","Data":"707d7b64202369487ef2689f2d01880e28058e64db38d0f3d6d3159890e685ad"} Jul 07 11:13:45.658405 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:45.658366 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"67e0c033-c10d-4952-883c-56e13a8f2e23","Type":"ContainerStarted","Data":"629362cd5530dcd19e907aa1a51cc921bb23fc821cbbc1f2a583f1f53289b1d6"} Jul 07 11:13:45.767536 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:45.767508 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c088023f-86df-4531-be68-1a21bad47ae6" path="/var/lib/kubelet/pods/c088023f-86df-4531-be68-1a21bad47ae6/volumes" Jul 07 11:13:46.674330 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:46.674294 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"67e0c033-c10d-4952-883c-56e13a8f2e23","Type":"ContainerStarted","Data":"48637b4ba7454f5ed91206878ea75adce0e980964a3f3694fc64de168656d670"} Jul 07 11:13:46.674701 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:46.674380 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"67e0c033-c10d-4952-883c-56e13a8f2e23","Type":"ContainerStarted","Data":"5e7ae4294c90e98f5598cca2b946bb6542b691243e6f21687f7daa44edab57a1"} Jul 07 11:13:46.674701 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:46.674405 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"67e0c033-c10d-4952-883c-56e13a8f2e23","Type":"ContainerStarted","Data":"f3deae6fe628ab2f79f90d97e45d4a02e3ef0398f4445cc2038990ff49069540"} Jul 07 11:13:46.674701 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:46.674434 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"67e0c033-c10d-4952-883c-56e13a8f2e23","Type":"ContainerStarted","Data":"f2efbf2a25f2a3021799169c2543b9d3b887518d27563b940f870e625d81c053"} Jul 07 11:13:46.674701 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:46.674458 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"67e0c033-c10d-4952-883c-56e13a8f2e23","Type":"ContainerStarted","Data":"4c7307deaa302b9cba9b8a96ee7276a49167a060b4f26376cb69363f32621021"} Jul 07 11:13:46.674701 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:46.674481 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"67e0c033-c10d-4952-883c-56e13a8f2e23","Type":"ContainerStarted","Data":"c7ed273d8c1ea510b36020cdef921aa4ac71adcb46fc46f6613b99e63e5813a5"} Jul 07 11:13:46.703248 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:46.703189 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.684863925 podStartE2EDuration="2.703171065s" totalImagesPullingTime="0s" totalInitContainerRuntime="18.30714ms" isStatefulPod=false podCreationTimestamp="2026-07-07 11:13:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:13:46.701564148 +0000 UTC m=+151.115433327" watchObservedRunningTime="2026-07-07 11:13:46.703171065 +0000 UTC m=+151.117040230" Jul 07 11:13:46.889475 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:46.889443 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:13:46.943251 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:46.943187 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:13:50.034660 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:50.034628 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:13:56.073564 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:56.073527 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:13:56.175177 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:56.175140 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:13:56.252408 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:56.252368 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:13:57.590188 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:57.590149 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:13:57.686459 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:13:57.686418 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:14:12.320454 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:14:12.320423 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:14:30.933411 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:14:30.933358 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:14:31.014267 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:14:31.014238 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:14:33.081753 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:14:33.081718 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:14:33.254793 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:14:33.254755 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:14:33.323533 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:14:33.323503 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:14:34.480473 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:14:34.480435 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:14:34.580316 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:14:34.580275 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:14:45.035311 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:14:45.035283 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:14:45.049908 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:14:45.049885 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:14:45.939839 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:14:45.939805 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 11:15:01.585072 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:15:01.585035 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:15:02.470868 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:15:02.470826 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:15:05.696593 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:15:05.696555 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:15:05.812234 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:15:05.812197 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:15:07.115368 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:15:07.115336 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:15:07.439336 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:15:07.439302 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:15:07.457303 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:15:07.457274 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-kn6g2"] Jul 07 11:15:07.459537 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:15:07.459516 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-kn6g2" Jul 07 11:15:07.467902 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:15:07.467880 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 07 11:15:07.468141 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:15:07.468128 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 07 11:15:07.483531 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:15:07.483505 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4bbad287-5075-4941-86b2-7a0893abf3d0-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-kn6g2\" (UID: \"4bbad287-5075-4941-86b2-7a0893abf3d0\") " pod="cert-manager/cert-manager-webhook-6486ffd997-kn6g2" Jul 07 11:15:07.483650 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:15:07.483635 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4l9kn\" (UniqueName: \"kubernetes.io/projected/4bbad287-5075-4941-86b2-7a0893abf3d0-kube-api-access-4l9kn\") pod \"cert-manager-webhook-6486ffd997-kn6g2\" (UID: \"4bbad287-5075-4941-86b2-7a0893abf3d0\") " pod="cert-manager/cert-manager-webhook-6486ffd997-kn6g2" Jul 07 11:15:07.494282 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:15:07.494258 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-kn6g2"] Jul 07 11:15:07.584176 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:15:07.584140 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4l9kn\" (UniqueName: \"kubernetes.io/projected/4bbad287-5075-4941-86b2-7a0893abf3d0-kube-api-access-4l9kn\") pod \"cert-manager-webhook-6486ffd997-kn6g2\" (UID: \"4bbad287-5075-4941-86b2-7a0893abf3d0\") " pod="cert-manager/cert-manager-webhook-6486ffd997-kn6g2" Jul 07 11:15:07.584386 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:15:07.584188 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4bbad287-5075-4941-86b2-7a0893abf3d0-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-kn6g2\" (UID: \"4bbad287-5075-4941-86b2-7a0893abf3d0\") " pod="cert-manager/cert-manager-webhook-6486ffd997-kn6g2" Jul 07 11:15:07.607801 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:15:07.607771 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4bbad287-5075-4941-86b2-7a0893abf3d0-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-kn6g2\" (UID: \"4bbad287-5075-4941-86b2-7a0893abf3d0\") " pod="cert-manager/cert-manager-webhook-6486ffd997-kn6g2" Jul 07 11:15:07.618432 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:15:07.618399 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4l9kn\" (UniqueName: \"kubernetes.io/projected/4bbad287-5075-4941-86b2-7a0893abf3d0-kube-api-access-4l9kn\") pod \"cert-manager-webhook-6486ffd997-kn6g2\" (UID: \"4bbad287-5075-4941-86b2-7a0893abf3d0\") " pod="cert-manager/cert-manager-webhook-6486ffd997-kn6g2" Jul 07 11:15:07.789900 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:15:07.789818 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-kn6g2" Jul 07 11:15:07.932848 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:15:07.932815 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:15:07.956603 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:15:07.956583 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-kn6g2"] Jul 07 11:15:07.959327 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:15:07.959299 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4bbad287_5075_4941_86b2_7a0893abf3d0.slice/crio-70b6f9d76bf36f66f77100a422a1903392ee049880a99373f029fbe65aa36c11 WatchSource:0}: Error finding container 70b6f9d76bf36f66f77100a422a1903392ee049880a99373f029fbe65aa36c11: Status 404 returned error can't find the container with id 70b6f9d76bf36f66f77100a422a1903392ee049880a99373f029fbe65aa36c11 Jul 07 11:15:08.015145 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:15:08.015111 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-kn6g2" event={"ID":"4bbad287-5075-4941-86b2-7a0893abf3d0","Type":"ContainerStarted","Data":"70b6f9d76bf36f66f77100a422a1903392ee049880a99373f029fbe65aa36c11"} Jul 07 11:15:08.165847 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:15:08.165810 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:15:09.192223 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:15:09.192156 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:15:09.661331 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:15:09.661283 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:15:09.970405 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:15:09.970327 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-kxtnl"] Jul 07 11:15:09.974277 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:15:09.973982 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-kxtnl" Jul 07 11:15:09.978394 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:15:09.978376 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-p6vqm\"" Jul 07 11:15:10.001898 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:15:10.001864 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-kxtnl"] Jul 07 11:15:10.005601 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:15:10.005574 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/24179cff-6d1a-4c61-8f99-085f49b8759f-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-kxtnl\" (UID: \"24179cff-6d1a-4c61-8f99-085f49b8759f\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-kxtnl" Jul 07 11:15:10.005733 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:15:10.005625 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x58pg\" (UniqueName: \"kubernetes.io/projected/24179cff-6d1a-4c61-8f99-085f49b8759f-kube-api-access-x58pg\") pod \"cert-manager-cainjector-559cdd8c87-kxtnl\" (UID: \"24179cff-6d1a-4c61-8f99-085f49b8759f\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-kxtnl" Jul 07 11:15:10.106705 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:15:10.106666 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/24179cff-6d1a-4c61-8f99-085f49b8759f-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-kxtnl\" (UID: \"24179cff-6d1a-4c61-8f99-085f49b8759f\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-kxtnl" Jul 07 11:15:10.106889 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:15:10.106721 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x58pg\" (UniqueName: \"kubernetes.io/projected/24179cff-6d1a-4c61-8f99-085f49b8759f-kube-api-access-x58pg\") pod \"cert-manager-cainjector-559cdd8c87-kxtnl\" (UID: \"24179cff-6d1a-4c61-8f99-085f49b8759f\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-kxtnl" Jul 07 11:15:10.117640 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:15:10.117607 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/24179cff-6d1a-4c61-8f99-085f49b8759f-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-kxtnl\" (UID: \"24179cff-6d1a-4c61-8f99-085f49b8759f\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-kxtnl" Jul 07 11:15:10.120141 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:15:10.120113 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x58pg\" (UniqueName: \"kubernetes.io/projected/24179cff-6d1a-4c61-8f99-085f49b8759f-kube-api-access-x58pg\") pod \"cert-manager-cainjector-559cdd8c87-kxtnl\" (UID: \"24179cff-6d1a-4c61-8f99-085f49b8759f\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-kxtnl" Jul 07 11:15:10.284738 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:15:10.284645 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-kxtnl" Jul 07 11:15:11.009706 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:15:11.009631 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-kxtnl"] Jul 07 11:15:11.012278 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:15:11.012257 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod24179cff_6d1a_4c61_8f99_085f49b8759f.slice/crio-c44ad5252c5813f33b57ab0a9c79860fe3a1f17d92b12f3e450b981e467193a6 WatchSource:0}: Error finding container c44ad5252c5813f33b57ab0a9c79860fe3a1f17d92b12f3e450b981e467193a6: Status 404 returned error can't find the container with id c44ad5252c5813f33b57ab0a9c79860fe3a1f17d92b12f3e450b981e467193a6 Jul 07 11:15:11.031682 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:15:11.031651 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-kxtnl" event={"ID":"24179cff-6d1a-4c61-8f99-085f49b8759f","Type":"ContainerStarted","Data":"c44ad5252c5813f33b57ab0a9c79860fe3a1f17d92b12f3e450b981e467193a6"} Jul 07 11:15:11.033437 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:15:11.033408 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-kn6g2" event={"ID":"4bbad287-5075-4941-86b2-7a0893abf3d0","Type":"ContainerStarted","Data":"29f6b6389b416208ee50d0ad14ada91772c776a32a35b3c271008156bb28eefa"} Jul 07 11:15:11.033547 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:15:11.033533 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-kn6g2" Jul 07 11:15:11.064983 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:15:11.064938 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-kn6g2" podStartSLOduration=1.113837359 podStartE2EDuration="4.064894439s" totalImagesPullingTime="2.95105708s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:15:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:15:11.060959662 +0000 UTC m=+235.474828827" watchObservedRunningTime="2026-07-07 11:15:11.064894439 +0000 UTC m=+235.478763604" Jul 07 11:15:12.039503 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:15:12.039471 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-kxtnl" event={"ID":"24179cff-6d1a-4c61-8f99-085f49b8759f","Type":"ContainerStarted","Data":"78c14d8e814a01ed5ee240af9e8367e45bc075a20eb2cb53428cde157886d2fd"} Jul 07 11:15:12.067317 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:15:12.067263 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-kxtnl" podStartSLOduration=3.067245398 podStartE2EDuration="3.067245398s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:15:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:15:12.06611384 +0000 UTC m=+236.479983005" watchObservedRunningTime="2026-07-07 11:15:12.067245398 +0000 UTC m=+236.481114563" Jul 07 11:15:12.292773 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:15:12.292671 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:15:12.383793 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:15:12.383762 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:15:17.041672 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:15:17.041645 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-kn6g2" Jul 07 11:15:17.181440 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:15:17.181390 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:15:19.671096 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:15:19.671062 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:15:19.813072 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:15:19.813030 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:15:24.967181 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:15:24.967143 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:15:25.169342 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:15:25.169307 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:15:27.063001 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:15:27.062968 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:15:37.610975 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:15:37.610939 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:15:37.726345 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:15:37.726311 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:15:38.818217 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:15:38.818137 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:15:38.949387 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:15:38.949350 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:15:39.396478 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:15:39.396447 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:15:39.932167 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:15:39.932128 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:15:40.103318 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:15:40.103290 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:15:40.130601 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:15:40.130576 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6dc7dd79cd-wxb6p"] Jul 07 11:15:40.134310 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:15:40.134292 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6dc7dd79cd-wxb6p" Jul 07 11:15:40.139905 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:15:40.139884 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 07 11:15:40.140361 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:15:40.140343 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 07 11:15:40.140786 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:15:40.140490 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 07 11:15:40.140786 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:15:40.140525 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-6xcdv\"" Jul 07 11:15:40.140786 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:15:40.140651 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 07 11:15:40.183673 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:15:40.183615 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6dc7dd79cd-wxb6p"] Jul 07 11:15:40.259250 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:15:40.259226 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/8ccc2735-03a2-45ed-94c6-035b5288903c-manifests\") pod \"opendatahub-operator-controller-manager-6dc7dd79cd-wxb6p\" (UID: \"8ccc2735-03a2-45ed-94c6-035b5288903c\") " pod="opendatahub/opendatahub-operator-controller-manager-6dc7dd79cd-wxb6p" Jul 07 11:15:40.259347 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:15:40.259283 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8ccc2735-03a2-45ed-94c6-035b5288903c-webhook-cert\") pod \"opendatahub-operator-controller-manager-6dc7dd79cd-wxb6p\" (UID: \"8ccc2735-03a2-45ed-94c6-035b5288903c\") " pod="opendatahub/opendatahub-operator-controller-manager-6dc7dd79cd-wxb6p" Jul 07 11:15:40.259388 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:15:40.259345 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/8ccc2735-03a2-45ed-94c6-035b5288903c-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6dc7dd79cd-wxb6p\" (UID: \"8ccc2735-03a2-45ed-94c6-035b5288903c\") " pod="opendatahub/opendatahub-operator-controller-manager-6dc7dd79cd-wxb6p" Jul 07 11:15:40.259388 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:15:40.259376 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8ccc2735-03a2-45ed-94c6-035b5288903c-tmp\") pod \"opendatahub-operator-controller-manager-6dc7dd79cd-wxb6p\" (UID: \"8ccc2735-03a2-45ed-94c6-035b5288903c\") " pod="opendatahub/opendatahub-operator-controller-manager-6dc7dd79cd-wxb6p" Jul 07 11:15:40.259453 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:15:40.259408 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wqt2w\" (UniqueName: \"kubernetes.io/projected/8ccc2735-03a2-45ed-94c6-035b5288903c-kube-api-access-wqt2w\") pod \"opendatahub-operator-controller-manager-6dc7dd79cd-wxb6p\" (UID: \"8ccc2735-03a2-45ed-94c6-035b5288903c\") " pod="opendatahub/opendatahub-operator-controller-manager-6dc7dd79cd-wxb6p" Jul 07 11:15:40.360734 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:15:40.360708 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wqt2w\" (UniqueName: \"kubernetes.io/projected/8ccc2735-03a2-45ed-94c6-035b5288903c-kube-api-access-wqt2w\") pod \"opendatahub-operator-controller-manager-6dc7dd79cd-wxb6p\" (UID: \"8ccc2735-03a2-45ed-94c6-035b5288903c\") " pod="opendatahub/opendatahub-operator-controller-manager-6dc7dd79cd-wxb6p" Jul 07 11:15:40.360824 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:15:40.360743 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/8ccc2735-03a2-45ed-94c6-035b5288903c-manifests\") pod \"opendatahub-operator-controller-manager-6dc7dd79cd-wxb6p\" (UID: \"8ccc2735-03a2-45ed-94c6-035b5288903c\") " pod="opendatahub/opendatahub-operator-controller-manager-6dc7dd79cd-wxb6p" Jul 07 11:15:40.360824 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:15:40.360792 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8ccc2735-03a2-45ed-94c6-035b5288903c-webhook-cert\") pod \"opendatahub-operator-controller-manager-6dc7dd79cd-wxb6p\" (UID: \"8ccc2735-03a2-45ed-94c6-035b5288903c\") " pod="opendatahub/opendatahub-operator-controller-manager-6dc7dd79cd-wxb6p" Jul 07 11:15:40.360824 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:15:40.360817 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/8ccc2735-03a2-45ed-94c6-035b5288903c-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6dc7dd79cd-wxb6p\" (UID: \"8ccc2735-03a2-45ed-94c6-035b5288903c\") " pod="opendatahub/opendatahub-operator-controller-manager-6dc7dd79cd-wxb6p" Jul 07 11:15:40.360996 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:15:40.360836 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8ccc2735-03a2-45ed-94c6-035b5288903c-tmp\") pod \"opendatahub-operator-controller-manager-6dc7dd79cd-wxb6p\" (UID: \"8ccc2735-03a2-45ed-94c6-035b5288903c\") " pod="opendatahub/opendatahub-operator-controller-manager-6dc7dd79cd-wxb6p" Jul 07 11:15:40.361302 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:15:40.361283 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/8ccc2735-03a2-45ed-94c6-035b5288903c-manifests\") pod \"opendatahub-operator-controller-manager-6dc7dd79cd-wxb6p\" (UID: \"8ccc2735-03a2-45ed-94c6-035b5288903c\") " pod="opendatahub/opendatahub-operator-controller-manager-6dc7dd79cd-wxb6p" Jul 07 11:15:40.363151 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:15:40.363129 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8ccc2735-03a2-45ed-94c6-035b5288903c-tmp\") pod \"opendatahub-operator-controller-manager-6dc7dd79cd-wxb6p\" (UID: \"8ccc2735-03a2-45ed-94c6-035b5288903c\") " pod="opendatahub/opendatahub-operator-controller-manager-6dc7dd79cd-wxb6p" Jul 07 11:15:40.363268 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:15:40.363250 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8ccc2735-03a2-45ed-94c6-035b5288903c-webhook-cert\") pod \"opendatahub-operator-controller-manager-6dc7dd79cd-wxb6p\" (UID: \"8ccc2735-03a2-45ed-94c6-035b5288903c\") " pod="opendatahub/opendatahub-operator-controller-manager-6dc7dd79cd-wxb6p" Jul 07 11:15:40.363403 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:15:40.363387 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/8ccc2735-03a2-45ed-94c6-035b5288903c-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6dc7dd79cd-wxb6p\" (UID: \"8ccc2735-03a2-45ed-94c6-035b5288903c\") " pod="opendatahub/opendatahub-operator-controller-manager-6dc7dd79cd-wxb6p" Jul 07 11:15:40.383443 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:15:40.383417 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:15:40.399560 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:15:40.399537 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wqt2w\" (UniqueName: \"kubernetes.io/projected/8ccc2735-03a2-45ed-94c6-035b5288903c-kube-api-access-wqt2w\") pod \"opendatahub-operator-controller-manager-6dc7dd79cd-wxb6p\" (UID: \"8ccc2735-03a2-45ed-94c6-035b5288903c\") " pod="opendatahub/opendatahub-operator-controller-manager-6dc7dd79cd-wxb6p" Jul 07 11:15:40.444281 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:15:40.444221 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6dc7dd79cd-wxb6p" Jul 07 11:15:40.564081 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:15:40.564046 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:15:40.587544 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:15:40.587517 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6dc7dd79cd-wxb6p"] Jul 07 11:15:40.590841 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:15:40.590818 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ccc2735_03a2_45ed_94c6_035b5288903c.slice/crio-e0d4393af9d690aff01f9661526498ce978c5fd1698e45a9c3f56ee88969e0a8 WatchSource:0}: Error finding container e0d4393af9d690aff01f9661526498ce978c5fd1698e45a9c3f56ee88969e0a8: Status 404 returned error can't find the container with id e0d4393af9d690aff01f9661526498ce978c5fd1698e45a9c3f56ee88969e0a8 Jul 07 11:15:40.680470 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:15:40.680443 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:15:40.779769 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:15:40.779714 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:15:40.880381 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:15:40.880338 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:15:41.159329 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:15:41.159302 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6dc7dd79cd-wxb6p" event={"ID":"8ccc2735-03a2-45ed-94c6-035b5288903c","Type":"ContainerStarted","Data":"e0d4393af9d690aff01f9661526498ce978c5fd1698e45a9c3f56ee88969e0a8"} Jul 07 11:15:44.175942 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:15:44.175898 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6dc7dd79cd-wxb6p" event={"ID":"8ccc2735-03a2-45ed-94c6-035b5288903c","Type":"ContainerStarted","Data":"9e051db8851c2de99d51203a12c0cf34396dda56dadc44ccd8279682db26fa25"} Jul 07 11:15:45.181605 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:15:45.181572 2562 generic.go:356] "Generic (PLEG): container finished" podID="8ccc2735-03a2-45ed-94c6-035b5288903c" containerID="9e051db8851c2de99d51203a12c0cf34396dda56dadc44ccd8279682db26fa25" exitCode=0 Jul 07 11:15:45.182031 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:15:45.181659 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6dc7dd79cd-wxb6p" event={"ID":"8ccc2735-03a2-45ed-94c6-035b5288903c","Type":"ContainerDied","Data":"9e051db8851c2de99d51203a12c0cf34396dda56dadc44ccd8279682db26fa25"} Jul 07 11:15:46.192362 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:15:46.192330 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6dc7dd79cd-wxb6p" event={"ID":"8ccc2735-03a2-45ed-94c6-035b5288903c","Type":"ContainerStarted","Data":"9e7ece34ff9f0a0930f8eaf01174f97aa510aa427162353ef719f0684582e49f"} Jul 07 11:15:46.192869 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:15:46.192454 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6dc7dd79cd-wxb6p" Jul 07 11:15:46.223964 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:15:46.223893 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6dc7dd79cd-wxb6p" podStartSLOduration=1.560050548 podStartE2EDuration="6.223878017s" totalImagesPullingTime="3.615664217s" totalInitContainerRuntime="1.048163252s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:15:40 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:15:46.221172195 +0000 UTC m=+270.635041359" watchObservedRunningTime="2026-07-07 11:15:46.223878017 +0000 UTC m=+270.637747181" Jul 07 11:15:49.467521 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:15:49.467461 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:15:49.708948 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:15:49.708877 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:15:57.199523 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:15:57.199490 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6dc7dd79cd-wxb6p" Jul 07 11:15:59.013962 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:15:59.013898 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:16:00.110302 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:16:00.110264 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-7769b86749-cdjb9"] Jul 07 11:16:00.120910 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:16:00.120883 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7769b86749-cdjb9" Jul 07 11:16:00.129837 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:16:00.129538 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-z4bzt\"" Jul 07 11:16:00.129837 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:16:00.129602 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 07 11:16:00.130019 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:16:00.129845 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 07 11:16:00.130346 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:16:00.130086 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 07 11:16:00.130447 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:16:00.130377 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 07 11:16:00.130506 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:16:00.130486 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 07 11:16:00.131388 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:16:00.131367 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7769b86749-cdjb9"] Jul 07 11:16:00.221365 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:16:00.221332 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p46nv\" (UniqueName: \"kubernetes.io/projected/eb008980-137b-4f0f-b31c-b1c5db989c50-kube-api-access-p46nv\") pod \"lws-controller-manager-7769b86749-cdjb9\" (UID: \"eb008980-137b-4f0f-b31c-b1c5db989c50\") " pod="openshift-lws-operator/lws-controller-manager-7769b86749-cdjb9" Jul 07 11:16:00.221543 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:16:00.221390 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/eb008980-137b-4f0f-b31c-b1c5db989c50-metrics-cert\") pod \"lws-controller-manager-7769b86749-cdjb9\" (UID: \"eb008980-137b-4f0f-b31c-b1c5db989c50\") " pod="openshift-lws-operator/lws-controller-manager-7769b86749-cdjb9" Jul 07 11:16:00.221543 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:16:00.221425 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/eb008980-137b-4f0f-b31c-b1c5db989c50-manager-config\") pod \"lws-controller-manager-7769b86749-cdjb9\" (UID: \"eb008980-137b-4f0f-b31c-b1c5db989c50\") " pod="openshift-lws-operator/lws-controller-manager-7769b86749-cdjb9" Jul 07 11:16:00.221543 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:16:00.221487 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/eb008980-137b-4f0f-b31c-b1c5db989c50-cert\") pod \"lws-controller-manager-7769b86749-cdjb9\" (UID: \"eb008980-137b-4f0f-b31c-b1c5db989c50\") " pod="openshift-lws-operator/lws-controller-manager-7769b86749-cdjb9" Jul 07 11:16:00.322147 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:16:00.322113 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/eb008980-137b-4f0f-b31c-b1c5db989c50-metrics-cert\") pod \"lws-controller-manager-7769b86749-cdjb9\" (UID: \"eb008980-137b-4f0f-b31c-b1c5db989c50\") " pod="openshift-lws-operator/lws-controller-manager-7769b86749-cdjb9" Jul 07 11:16:00.322332 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:16:00.322164 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/eb008980-137b-4f0f-b31c-b1c5db989c50-manager-config\") pod \"lws-controller-manager-7769b86749-cdjb9\" (UID: \"eb008980-137b-4f0f-b31c-b1c5db989c50\") " pod="openshift-lws-operator/lws-controller-manager-7769b86749-cdjb9" Jul 07 11:16:00.322332 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:16:00.322197 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/eb008980-137b-4f0f-b31c-b1c5db989c50-cert\") pod \"lws-controller-manager-7769b86749-cdjb9\" (UID: \"eb008980-137b-4f0f-b31c-b1c5db989c50\") " pod="openshift-lws-operator/lws-controller-manager-7769b86749-cdjb9" Jul 07 11:16:00.322332 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:16:00.322229 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p46nv\" (UniqueName: \"kubernetes.io/projected/eb008980-137b-4f0f-b31c-b1c5db989c50-kube-api-access-p46nv\") pod \"lws-controller-manager-7769b86749-cdjb9\" (UID: \"eb008980-137b-4f0f-b31c-b1c5db989c50\") " pod="openshift-lws-operator/lws-controller-manager-7769b86749-cdjb9" Jul 07 11:16:00.323049 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:16:00.323021 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/eb008980-137b-4f0f-b31c-b1c5db989c50-manager-config\") pod \"lws-controller-manager-7769b86749-cdjb9\" (UID: \"eb008980-137b-4f0f-b31c-b1c5db989c50\") " pod="openshift-lws-operator/lws-controller-manager-7769b86749-cdjb9" Jul 07 11:16:00.324664 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:16:00.324633 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/eb008980-137b-4f0f-b31c-b1c5db989c50-cert\") pod \"lws-controller-manager-7769b86749-cdjb9\" (UID: \"eb008980-137b-4f0f-b31c-b1c5db989c50\") " pod="openshift-lws-operator/lws-controller-manager-7769b86749-cdjb9" Jul 07 11:16:00.324762 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:16:00.324665 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/eb008980-137b-4f0f-b31c-b1c5db989c50-metrics-cert\") pod \"lws-controller-manager-7769b86749-cdjb9\" (UID: \"eb008980-137b-4f0f-b31c-b1c5db989c50\") " pod="openshift-lws-operator/lws-controller-manager-7769b86749-cdjb9" Jul 07 11:16:00.332573 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:16:00.332550 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p46nv\" (UniqueName: \"kubernetes.io/projected/eb008980-137b-4f0f-b31c-b1c5db989c50-kube-api-access-p46nv\") pod \"lws-controller-manager-7769b86749-cdjb9\" (UID: \"eb008980-137b-4f0f-b31c-b1c5db989c50\") " pod="openshift-lws-operator/lws-controller-manager-7769b86749-cdjb9" Jul 07 11:16:00.403560 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:16:00.403527 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:16:00.433199 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:16:00.433174 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7769b86749-cdjb9" Jul 07 11:16:00.511928 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:16:00.511882 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:16:00.608480 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:16:00.608450 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7769b86749-cdjb9"] Jul 07 11:16:00.619557 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:16:00.619526 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb008980_137b_4f0f_b31c_b1c5db989c50.slice/crio-c0c74025f4edbd27bd1593a62ff7873ac1c63c406fe50d36fc54a93cf941b776 WatchSource:0}: Error finding container c0c74025f4edbd27bd1593a62ff7873ac1c63c406fe50d36fc54a93cf941b776: Status 404 returned error can't find the container with id c0c74025f4edbd27bd1593a62ff7873ac1c63c406fe50d36fc54a93cf941b776 Jul 07 11:16:00.783799 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:16:00.783703 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:16:01.018933 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:16:01.018878 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:16:01.261366 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:16:01.261332 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7769b86749-cdjb9" event={"ID":"eb008980-137b-4f0f-b31c-b1c5db989c50","Type":"ContainerStarted","Data":"c0c74025f4edbd27bd1593a62ff7873ac1c63c406fe50d36fc54a93cf941b776"} Jul 07 11:16:03.273789 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:16:03.273746 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7769b86749-cdjb9" event={"ID":"eb008980-137b-4f0f-b31c-b1c5db989c50","Type":"ContainerStarted","Data":"90db80358751d7e452a89080e2b8725b58e9e063f4aacdce3d563e0512af788e"} Jul 07 11:16:03.274289 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:16:03.274032 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-7769b86749-cdjb9" Jul 07 11:16:03.307813 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:16:03.307749 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-7769b86749-cdjb9" podStartSLOduration=0.755745604 podStartE2EDuration="3.307728356s" totalImagesPullingTime="2.551982752s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:16:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:16:03.306492313 +0000 UTC m=+287.720361478" watchObservedRunningTime="2026-07-07 11:16:03.307728356 +0000 UTC m=+287.721597521" Jul 07 11:16:05.385000 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:16:05.384959 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:16:05.463230 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:16:05.463201 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-5df5f4cc47-4ckp9"] Jul 07 11:16:05.466697 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:16:05.466678 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5df5f4cc47-4ckp9" Jul 07 11:16:05.470544 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:16:05.470511 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 07 11:16:05.470544 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:16:05.470528 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-nfpl6\"" Jul 07 11:16:05.470544 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:16:05.470517 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 07 11:16:05.471274 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:16:05.471254 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 07 11:16:05.480594 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:16:05.480570 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 07 11:16:05.491213 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:16:05.491189 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5df5f4cc47-4ckp9"] Jul 07 11:16:05.571644 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:16:05.571597 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f0e9dcfd-d744-4b75-8186-bbbd8c43bad2-tls-certs\") pod \"kube-auth-proxy-5df5f4cc47-4ckp9\" (UID: \"f0e9dcfd-d744-4b75-8186-bbbd8c43bad2\") " pod="openshift-ingress/kube-auth-proxy-5df5f4cc47-4ckp9" Jul 07 11:16:05.571644 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:16:05.571638 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f0e9dcfd-d744-4b75-8186-bbbd8c43bad2-tmp\") pod \"kube-auth-proxy-5df5f4cc47-4ckp9\" (UID: \"f0e9dcfd-d744-4b75-8186-bbbd8c43bad2\") " pod="openshift-ingress/kube-auth-proxy-5df5f4cc47-4ckp9" Jul 07 11:16:05.571892 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:16:05.571663 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l2dtx\" (UniqueName: \"kubernetes.io/projected/f0e9dcfd-d744-4b75-8186-bbbd8c43bad2-kube-api-access-l2dtx\") pod \"kube-auth-proxy-5df5f4cc47-4ckp9\" (UID: \"f0e9dcfd-d744-4b75-8186-bbbd8c43bad2\") " pod="openshift-ingress/kube-auth-proxy-5df5f4cc47-4ckp9" Jul 07 11:16:05.673060 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:16:05.672973 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f0e9dcfd-d744-4b75-8186-bbbd8c43bad2-tls-certs\") pod \"kube-auth-proxy-5df5f4cc47-4ckp9\" (UID: \"f0e9dcfd-d744-4b75-8186-bbbd8c43bad2\") " pod="openshift-ingress/kube-auth-proxy-5df5f4cc47-4ckp9" Jul 07 11:16:05.673060 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:16:05.673023 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f0e9dcfd-d744-4b75-8186-bbbd8c43bad2-tmp\") pod \"kube-auth-proxy-5df5f4cc47-4ckp9\" (UID: \"f0e9dcfd-d744-4b75-8186-bbbd8c43bad2\") " pod="openshift-ingress/kube-auth-proxy-5df5f4cc47-4ckp9" Jul 07 11:16:05.673243 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:16:05.673118 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l2dtx\" (UniqueName: \"kubernetes.io/projected/f0e9dcfd-d744-4b75-8186-bbbd8c43bad2-kube-api-access-l2dtx\") pod \"kube-auth-proxy-5df5f4cc47-4ckp9\" (UID: \"f0e9dcfd-d744-4b75-8186-bbbd8c43bad2\") " pod="openshift-ingress/kube-auth-proxy-5df5f4cc47-4ckp9" Jul 07 11:16:05.673243 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:16:05.673137 2562 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found Jul 07 11:16:05.673243 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:16:05.673219 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f0e9dcfd-d744-4b75-8186-bbbd8c43bad2-tls-certs podName:f0e9dcfd-d744-4b75-8186-bbbd8c43bad2 nodeName:}" failed. No retries permitted until 2026-07-07 11:16:06.173196662 +0000 UTC m=+290.587065822 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/f0e9dcfd-d744-4b75-8186-bbbd8c43bad2-tls-certs") pod "kube-auth-proxy-5df5f4cc47-4ckp9" (UID: "f0e9dcfd-d744-4b75-8186-bbbd8c43bad2") : secret "kube-auth-proxy-tls" not found Jul 07 11:16:05.675295 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:16:05.675276 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f0e9dcfd-d744-4b75-8186-bbbd8c43bad2-tmp\") pod \"kube-auth-proxy-5df5f4cc47-4ckp9\" (UID: \"f0e9dcfd-d744-4b75-8186-bbbd8c43bad2\") " pod="openshift-ingress/kube-auth-proxy-5df5f4cc47-4ckp9" Jul 07 11:16:05.689909 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:16:05.689885 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l2dtx\" (UniqueName: \"kubernetes.io/projected/f0e9dcfd-d744-4b75-8186-bbbd8c43bad2-kube-api-access-l2dtx\") pod \"kube-auth-proxy-5df5f4cc47-4ckp9\" (UID: \"f0e9dcfd-d744-4b75-8186-bbbd8c43bad2\") " pod="openshift-ingress/kube-auth-proxy-5df5f4cc47-4ckp9" Jul 07 11:16:05.858205 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:16:05.858160 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:16:06.177203 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:16:06.177170 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f0e9dcfd-d744-4b75-8186-bbbd8c43bad2-tls-certs\") pod \"kube-auth-proxy-5df5f4cc47-4ckp9\" (UID: \"f0e9dcfd-d744-4b75-8186-bbbd8c43bad2\") " pod="openshift-ingress/kube-auth-proxy-5df5f4cc47-4ckp9" Jul 07 11:16:06.179475 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:16:06.179445 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f0e9dcfd-d744-4b75-8186-bbbd8c43bad2-tls-certs\") pod \"kube-auth-proxy-5df5f4cc47-4ckp9\" (UID: \"f0e9dcfd-d744-4b75-8186-bbbd8c43bad2\") " pod="openshift-ingress/kube-auth-proxy-5df5f4cc47-4ckp9" Jul 07 11:16:06.376837 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:16:06.376808 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5df5f4cc47-4ckp9" Jul 07 11:16:06.518163 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:16:06.518130 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5df5f4cc47-4ckp9"] Jul 07 11:16:06.520826 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:16:06.520797 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0e9dcfd_d744_4b75_8186_bbbd8c43bad2.slice/crio-296dfd3f25054593a9a33e98f689531471bc3000748b0e2685a904a8408466b9 WatchSource:0}: Error finding container 296dfd3f25054593a9a33e98f689531471bc3000748b0e2685a904a8408466b9: Status 404 returned error can't find the container with id 296dfd3f25054593a9a33e98f689531471bc3000748b0e2685a904a8408466b9 Jul 07 11:16:07.294865 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:16:07.294833 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5df5f4cc47-4ckp9" event={"ID":"f0e9dcfd-d744-4b75-8186-bbbd8c43bad2","Type":"ContainerStarted","Data":"296dfd3f25054593a9a33e98f689531471bc3000748b0e2685a904a8408466b9"} Jul 07 11:16:07.891100 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:16:07.891064 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:16:07.992157 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:16:07.992121 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:16:09.128366 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:16:09.128334 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:16:10.312853 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:16:10.312814 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5df5f4cc47-4ckp9" event={"ID":"f0e9dcfd-d744-4b75-8186-bbbd8c43bad2","Type":"ContainerStarted","Data":"7ca9a890aca24ac026c200fec683521809a29fd2471732cf4d8efe0d14491348"} Jul 07 11:16:10.350172 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:16:10.350120 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-5df5f4cc47-4ckp9" podStartSLOduration=2.076724782 podStartE2EDuration="5.350104643s" totalImagesPullingTime="3.273379861s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:16:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:16:10.347908816 +0000 UTC m=+294.761777982" watchObservedRunningTime="2026-07-07 11:16:10.350104643 +0000 UTC m=+294.763973844" Jul 07 11:16:11.111277 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:16:11.111239 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:16:11.268555 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:16:11.268525 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:16:11.395989 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:16:11.395939 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:16:11.658240 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:16:11.658168 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:16:12.244691 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:16:12.244655 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:16:12.340825 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:16:12.340795 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:16:12.448198 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:16:12.448162 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:16:12.628818 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:16:12.628782 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:16:13.419855 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:16:13.419819 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:16:13.533436 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:16:13.533402 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:16:13.676140 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:16:13.676056 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:16:14.281625 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:16:14.281598 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-7769b86749-cdjb9" Jul 07 11:16:15.946710 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:16:15.946679 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tzf62_d2340b4b-179a-4c0a-9879-13b49897c939/ovn-acl-logging/0.log" Jul 07 11:16:15.949712 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:16:15.949692 2562 kubelet.go:1648] "Image garbage collection succeeded" Jul 07 11:16:15.951746 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:16:15.951724 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tzf62_d2340b4b-179a-4c0a-9879-13b49897c939/ovn-acl-logging/0.log" Jul 07 11:16:35.729171 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:16:35.729123 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:17:00.651526 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:17:00.651491 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:17:00.799734 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:17:00.799695 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:17:00.924637 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:17:00.924557 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:17:28.681783 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:17:28.681750 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:17:32.105027 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:17:32.104980 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:17:32.952857 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:17:32.952822 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:17:36.005036 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:17:36.004965 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:17:36.158674 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:17:36.158642 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:17:36.808716 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:17:36.808668 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:17:37.943873 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:17:37.943831 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:17:39.931565 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:17:39.931515 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:17:40.452401 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:17:40.452377 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:17:43.368199 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:17:43.368168 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:17:45.708764 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:17:45.708728 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:17:45.821753 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:17:45.821716 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:17:49.500007 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:17:49.499969 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:17:49.586205 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:17:49.586175 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:17:49.684876 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:17:49.684847 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:17:49.784251 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:17:49.784189 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:17:49.912354 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:17:49.912334 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:17:50.001947 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:17:50.001901 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:17:52.679390 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:17:52.679353 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:17:52.779121 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:17:52.779086 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:17:55.111442 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:17:55.111404 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-rwrbw"] Jul 07 11:17:55.114926 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:17:55.114892 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-rwrbw" Jul 07 11:17:55.117133 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:17:55.117111 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Jul 07 11:17:55.118102 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:17:55.118076 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-wwlh8\"" Jul 07 11:17:55.118213 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:17:55.118103 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Jul 07 11:17:55.118213 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:17:55.118112 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 07 11:17:55.118213 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:17:55.118088 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 07 11:17:55.125619 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:17:55.125600 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-rwrbw"] Jul 07 11:17:55.142199 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:17:55.142177 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:17:55.194327 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:17:55.194302 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/8557e6db-fe7e-47a1-9700-da25802680eb-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-rwrbw\" (UID: \"8557e6db-fe7e-47a1-9700-da25802680eb\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-rwrbw" Jul 07 11:17:55.194433 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:17:55.194333 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/8557e6db-fe7e-47a1-9700-da25802680eb-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-rwrbw\" (UID: \"8557e6db-fe7e-47a1-9700-da25802680eb\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-rwrbw" Jul 07 11:17:55.194433 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:17:55.194420 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cqw94\" (UniqueName: \"kubernetes.io/projected/8557e6db-fe7e-47a1-9700-da25802680eb-kube-api-access-cqw94\") pod \"kuadrant-console-plugin-66b5774985-rwrbw\" (UID: \"8557e6db-fe7e-47a1-9700-da25802680eb\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-rwrbw" Jul 07 11:17:55.287407 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:17:55.287375 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:17:55.295537 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:17:55.295515 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/8557e6db-fe7e-47a1-9700-da25802680eb-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-rwrbw\" (UID: \"8557e6db-fe7e-47a1-9700-da25802680eb\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-rwrbw" Jul 07 11:17:55.295634 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:17:55.295617 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cqw94\" (UniqueName: \"kubernetes.io/projected/8557e6db-fe7e-47a1-9700-da25802680eb-kube-api-access-cqw94\") pod \"kuadrant-console-plugin-66b5774985-rwrbw\" (UID: \"8557e6db-fe7e-47a1-9700-da25802680eb\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-rwrbw" Jul 07 11:17:55.295687 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:17:55.295673 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/8557e6db-fe7e-47a1-9700-da25802680eb-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-rwrbw\" (UID: \"8557e6db-fe7e-47a1-9700-da25802680eb\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-rwrbw" Jul 07 11:17:55.296295 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:17:55.296277 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/8557e6db-fe7e-47a1-9700-da25802680eb-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-rwrbw\" (UID: \"8557e6db-fe7e-47a1-9700-da25802680eb\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-rwrbw" Jul 07 11:17:55.298082 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:17:55.298058 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/8557e6db-fe7e-47a1-9700-da25802680eb-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-rwrbw\" (UID: \"8557e6db-fe7e-47a1-9700-da25802680eb\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-rwrbw" Jul 07 11:17:55.315823 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:17:55.315792 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cqw94\" (UniqueName: \"kubernetes.io/projected/8557e6db-fe7e-47a1-9700-da25802680eb-kube-api-access-cqw94\") pod \"kuadrant-console-plugin-66b5774985-rwrbw\" (UID: \"8557e6db-fe7e-47a1-9700-da25802680eb\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-rwrbw" Jul 07 11:17:55.425369 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:17:55.425343 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-rwrbw" Jul 07 11:17:55.560723 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:17:55.560700 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-rwrbw"] Jul 07 11:17:55.562903 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:17:55.562875 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8557e6db_fe7e_47a1_9700_da25802680eb.slice/crio-3a966da72f68243da054b856080c9d2a7791756787e8c7ecb0807c4e52687dbf WatchSource:0}: Error finding container 3a966da72f68243da054b856080c9d2a7791756787e8c7ecb0807c4e52687dbf: Status 404 returned error can't find the container with id 3a966da72f68243da054b856080c9d2a7791756787e8c7ecb0807c4e52687dbf Jul 07 11:17:55.564343 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:17:55.564323 2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 11:17:55.809109 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:17:55.809026 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-rwrbw" event={"ID":"8557e6db-fe7e-47a1-9700-da25802680eb","Type":"ContainerStarted","Data":"3a966da72f68243da054b856080c9d2a7791756787e8c7ecb0807c4e52687dbf"} Jul 07 11:17:57.838096 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:17:57.838056 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:17:57.939402 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:17:57.939371 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:18:00.183492 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:18:00.183427 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:18:00.283718 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:18:00.283656 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:18:00.381431 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:18:00.381385 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:18:04.889469 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:18:04.889426 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:18:05.043447 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:18:05.043419 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:18:08.253575 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:18:08.253526 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:18:08.379331 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:18:08.379294 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:18:08.483162 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:18:08.483126 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:18:11.139455 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:18:11.139425 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:18:11.240705 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:18:11.240669 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:18:16.397363 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:18:16.397330 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:18:16.496583 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:18:16.496547 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:18:19.942357 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:18:19.942321 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-rwrbw" event={"ID":"8557e6db-fe7e-47a1-9700-da25802680eb","Type":"ContainerStarted","Data":"59d7eb7920dc1c22f4d266473550b2de5ea2e60c873aecaaa33d197f1ddad5b0"} Jul 07 11:18:35.588451 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:18:35.588414 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:18:36.682038 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:18:36.681992 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:18:37.292128 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:18:37.292087 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:18:37.400035 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:18:37.399998 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:18:37.502271 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:18:37.502240 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:18:37.775331 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:18:37.775278 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-rwrbw" podStartSLOduration=19.196144073 podStartE2EDuration="42.775263389s" totalImagesPullingTime="23.579119316s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:17:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:18:19.976316847 +0000 UTC m=+424.390186009" watchObservedRunningTime="2026-07-07 11:18:37.775263389 +0000 UTC m=+442.189132559" Jul 07 11:18:37.776877 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:18:37.776851 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sns5p"] Jul 07 11:18:37.786656 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:18:37.786637 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-sns5p" Jul 07 11:18:37.789034 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:18:37.789014 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 07 11:18:37.792730 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:18:37.792679 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sns5p"] Jul 07 11:18:37.806531 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:18:37.806509 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sns5p"] Jul 07 11:18:37.875347 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:18:37.875318 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2gtgm\" (UniqueName: \"kubernetes.io/projected/de71dd9a-d584-4c85-b592-4381ce0a591d-kube-api-access-2gtgm\") pod \"limitador-limitador-54d6d988d5-sns5p\" (UID: \"de71dd9a-d584-4c85-b592-4381ce0a591d\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-sns5p" Jul 07 11:18:37.875491 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:18:37.875362 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/de71dd9a-d584-4c85-b592-4381ce0a591d-config-file\") pod \"limitador-limitador-54d6d988d5-sns5p\" (UID: \"de71dd9a-d584-4c85-b592-4381ce0a591d\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-sns5p" Jul 07 11:18:37.976751 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:18:37.976688 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/de71dd9a-d584-4c85-b592-4381ce0a591d-config-file\") pod \"limitador-limitador-54d6d988d5-sns5p\" (UID: \"de71dd9a-d584-4c85-b592-4381ce0a591d\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-sns5p" Jul 07 11:18:37.976871 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:18:37.976799 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2gtgm\" (UniqueName: \"kubernetes.io/projected/de71dd9a-d584-4c85-b592-4381ce0a591d-kube-api-access-2gtgm\") pod \"limitador-limitador-54d6d988d5-sns5p\" (UID: \"de71dd9a-d584-4c85-b592-4381ce0a591d\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-sns5p" Jul 07 11:18:37.977382 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:18:37.977365 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/de71dd9a-d584-4c85-b592-4381ce0a591d-config-file\") pod \"limitador-limitador-54d6d988d5-sns5p\" (UID: \"de71dd9a-d584-4c85-b592-4381ce0a591d\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-sns5p" Jul 07 11:18:37.985736 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:18:37.985714 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2gtgm\" (UniqueName: \"kubernetes.io/projected/de71dd9a-d584-4c85-b592-4381ce0a591d-kube-api-access-2gtgm\") pod \"limitador-limitador-54d6d988d5-sns5p\" (UID: \"de71dd9a-d584-4c85-b592-4381ce0a591d\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-sns5p" Jul 07 11:18:38.096969 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:18:38.096880 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-sns5p" Jul 07 11:18:38.098689 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:18:38.098666 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-6qtww"] Jul 07 11:18:38.105794 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:18:38.105766 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-6qtww" Jul 07 11:18:38.109152 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:18:38.109128 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-6v8rx\"" Jul 07 11:18:38.110731 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:18:38.110697 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-6qtww"] Jul 07 11:18:38.131743 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:18:38.131715 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:18:38.179241 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:18:38.179200 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p2glk\" (UniqueName: \"kubernetes.io/projected/964655a9-9fe5-4790-b52f-21e6cb1706e7-kube-api-access-p2glk\") pod \"authorino-58cd955675-6qtww\" (UID: \"964655a9-9fe5-4790-b52f-21e6cb1706e7\") " pod="kuadrant-system/authorino-58cd955675-6qtww" Jul 07 11:18:38.206512 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:18:38.206481 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:18:38.235041 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:18:38.235021 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sns5p"] Jul 07 11:18:38.237352 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:18:38.237327 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde71dd9a_d584_4c85_b592_4381ce0a591d.slice/crio-6ccafb783c712b954aba1a5c8fcebdfaf5d8574d50a3e33bc95c7fc1333772f9 WatchSource:0}: Error finding container 6ccafb783c712b954aba1a5c8fcebdfaf5d8574d50a3e33bc95c7fc1333772f9: Status 404 returned error can't find the container with id 6ccafb783c712b954aba1a5c8fcebdfaf5d8574d50a3e33bc95c7fc1333772f9 Jul 07 11:18:38.280677 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:18:38.280651 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p2glk\" (UniqueName: \"kubernetes.io/projected/964655a9-9fe5-4790-b52f-21e6cb1706e7-kube-api-access-p2glk\") pod \"authorino-58cd955675-6qtww\" (UID: \"964655a9-9fe5-4790-b52f-21e6cb1706e7\") " pod="kuadrant-system/authorino-58cd955675-6qtww" Jul 07 11:18:38.288522 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:18:38.288495 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p2glk\" (UniqueName: \"kubernetes.io/projected/964655a9-9fe5-4790-b52f-21e6cb1706e7-kube-api-access-p2glk\") pod \"authorino-58cd955675-6qtww\" (UID: \"964655a9-9fe5-4790-b52f-21e6cb1706e7\") " pod="kuadrant-system/authorino-58cd955675-6qtww" Jul 07 11:18:38.431956 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:18:38.431906 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-6qtww" Jul 07 11:18:38.560365 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:18:38.560335 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-6qtww"] Jul 07 11:18:38.563334 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:18:38.563306 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod964655a9_9fe5_4790_b52f_21e6cb1706e7.slice/crio-0c6573f243db25b2c9cce66d0ae6cb99b847e1171f1c6ea3391dff76dc3f7fa8 WatchSource:0}: Error finding container 0c6573f243db25b2c9cce66d0ae6cb99b847e1171f1c6ea3391dff76dc3f7fa8: Status 404 returned error can't find the container with id 0c6573f243db25b2c9cce66d0ae6cb99b847e1171f1c6ea3391dff76dc3f7fa8 Jul 07 11:18:39.031243 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:18:39.031166 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-sns5p" event={"ID":"de71dd9a-d584-4c85-b592-4381ce0a591d","Type":"ContainerStarted","Data":"6ccafb783c712b954aba1a5c8fcebdfaf5d8574d50a3e33bc95c7fc1333772f9"} Jul 07 11:18:39.033214 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:18:39.033183 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-6qtww" event={"ID":"964655a9-9fe5-4790-b52f-21e6cb1706e7","Type":"ContainerStarted","Data":"0c6573f243db25b2c9cce66d0ae6cb99b847e1171f1c6ea3391dff76dc3f7fa8"} Jul 07 11:18:41.983556 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:18:41.983528 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-6qtww"] Jul 07 11:18:44.067064 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:18:44.066976 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-6qtww" event={"ID":"964655a9-9fe5-4790-b52f-21e6cb1706e7","Type":"ContainerStarted","Data":"f9f4a03db6825ba9c969658deba55caa0e5bc8aa7ebe6ec735ee057723daf267"} Jul 07 11:18:44.067064 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:18:44.067044 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-6qtww" podUID="964655a9-9fe5-4790-b52f-21e6cb1706e7" containerName="authorino" containerID="cri-o://f9f4a03db6825ba9c969658deba55caa0e5bc8aa7ebe6ec735ee057723daf267" gracePeriod=30 Jul 07 11:18:44.068725 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:18:44.068701 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-sns5p" event={"ID":"de71dd9a-d584-4c85-b592-4381ce0a591d","Type":"ContainerStarted","Data":"fc69e927b03021a2a28bad63579624e09778f9e58ab26bf58f239f771db35034"} Jul 07 11:18:44.068838 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:18:44.068766 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-sns5p" Jul 07 11:18:44.085611 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:18:44.085572 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-6qtww" podStartSLOduration=0.984471099 podStartE2EDuration="6.085559836s" totalImagesPullingTime="5.101088737s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:18:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:18:44.083991513 +0000 UTC m=+448.497860680" watchObservedRunningTime="2026-07-07 11:18:44.085559836 +0000 UTC m=+448.499429000" Jul 07 11:18:44.106853 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:18:44.106816 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-sns5p" podStartSLOduration=1.6729390990000002 podStartE2EDuration="7.106803083s" totalImagesPullingTime="5.433863984s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:18:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:18:44.105495513 +0000 UTC m=+448.519364678" watchObservedRunningTime="2026-07-07 11:18:44.106803083 +0000 UTC m=+448.520672299" Jul 07 11:18:44.343340 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:18:44.343316 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-6qtww" Jul 07 11:18:44.441755 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:18:44.441731 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p2glk\" (UniqueName: \"kubernetes.io/projected/964655a9-9fe5-4790-b52f-21e6cb1706e7-kube-api-access-p2glk\") pod \"964655a9-9fe5-4790-b52f-21e6cb1706e7\" (UID: \"964655a9-9fe5-4790-b52f-21e6cb1706e7\") " Jul 07 11:18:44.443730 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:18:44.443703 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/964655a9-9fe5-4790-b52f-21e6cb1706e7-kube-api-access-p2glk" (OuterVolumeSpecName: "kube-api-access-p2glk") pod "964655a9-9fe5-4790-b52f-21e6cb1706e7" (UID: "964655a9-9fe5-4790-b52f-21e6cb1706e7"). InnerVolumeSpecName "kube-api-access-p2glk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 11:18:44.543197 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:18:44.543175 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p2glk\" (UniqueName: \"kubernetes.io/projected/964655a9-9fe5-4790-b52f-21e6cb1706e7-kube-api-access-p2glk\") on node \"ip-10-0-136-15.ec2.internal\" DevicePath \"\"" Jul 07 11:18:45.075254 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:18:45.075183 2562 generic.go:356] "Generic (PLEG): container finished" podID="964655a9-9fe5-4790-b52f-21e6cb1706e7" containerID="f9f4a03db6825ba9c969658deba55caa0e5bc8aa7ebe6ec735ee057723daf267" exitCode=0 Jul 07 11:18:45.075254 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:18:45.075230 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-6qtww" Jul 07 11:18:45.075722 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:18:45.075264 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-6qtww" event={"ID":"964655a9-9fe5-4790-b52f-21e6cb1706e7","Type":"ContainerDied","Data":"f9f4a03db6825ba9c969658deba55caa0e5bc8aa7ebe6ec735ee057723daf267"} Jul 07 11:18:45.075722 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:18:45.075310 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-6qtww" event={"ID":"964655a9-9fe5-4790-b52f-21e6cb1706e7","Type":"ContainerDied","Data":"0c6573f243db25b2c9cce66d0ae6cb99b847e1171f1c6ea3391dff76dc3f7fa8"} Jul 07 11:18:45.075722 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:18:45.075334 2562 scope.go:117] "RemoveContainer" containerID="f9f4a03db6825ba9c969658deba55caa0e5bc8aa7ebe6ec735ee057723daf267" Jul 07 11:18:45.084391 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:18:45.084235 2562 scope.go:117] "RemoveContainer" containerID="f9f4a03db6825ba9c969658deba55caa0e5bc8aa7ebe6ec735ee057723daf267" Jul 07 11:18:45.084581 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:18:45.084551 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f9f4a03db6825ba9c969658deba55caa0e5bc8aa7ebe6ec735ee057723daf267\": container with ID starting with f9f4a03db6825ba9c969658deba55caa0e5bc8aa7ebe6ec735ee057723daf267 not found: ID does not exist" containerID="f9f4a03db6825ba9c969658deba55caa0e5bc8aa7ebe6ec735ee057723daf267" Jul 07 11:18:45.084666 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:18:45.084590 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f9f4a03db6825ba9c969658deba55caa0e5bc8aa7ebe6ec735ee057723daf267"} err="failed to get container status \"f9f4a03db6825ba9c969658deba55caa0e5bc8aa7ebe6ec735ee057723daf267\": rpc error: code = NotFound desc = could not find container \"f9f4a03db6825ba9c969658deba55caa0e5bc8aa7ebe6ec735ee057723daf267\": container with ID starting with f9f4a03db6825ba9c969658deba55caa0e5bc8aa7ebe6ec735ee057723daf267 not found: ID does not exist" Jul 07 11:18:45.101565 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:18:45.101541 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-6qtww"] Jul 07 11:18:45.107515 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:18:45.107492 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-6qtww"] Jul 07 11:18:45.767474 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:18:45.767446 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="964655a9-9fe5-4790-b52f-21e6cb1706e7" path="/var/lib/kubelet/pods/964655a9-9fe5-4790-b52f-21e6cb1706e7/volumes" Jul 07 11:18:55.076980 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:18:55.076953 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-sns5p" Jul 07 11:18:58.813612 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:18:58.813579 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:19:10.707245 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:19:10.707209 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:19:10.805068 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:19:10.805035 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:19:10.946445 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:19:10.946418 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:19:40.323587 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:19:40.323550 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:19:40.652501 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:19:40.652461 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:19:40.794653 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:19:40.794614 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:19:41.040606 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:19:41.040534 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["keycloak-system/maas-keycloak-0"] Jul 07 11:19:41.040893 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:19:41.040880 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="964655a9-9fe5-4790-b52f-21e6cb1706e7" containerName="authorino" Jul 07 11:19:41.040971 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:19:41.040894 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="964655a9-9fe5-4790-b52f-21e6cb1706e7" containerName="authorino" Jul 07 11:19:41.041014 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:19:41.040977 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="964655a9-9fe5-4790-b52f-21e6cb1706e7" containerName="authorino" Jul 07 11:19:41.042874 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:19:41.042852 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jul 07 11:19:41.045500 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:19:41.045478 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"default-dockercfg-hndvg\"" Jul 07 11:19:41.045947 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:19:41.045933 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"kube-root-ca.crt\"" Jul 07 11:19:41.046360 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:19:41.046344 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"maas-keycloak-initial-admin\"" Jul 07 11:19:41.046441 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:19:41.046360 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"openshift-service-ca.crt\"" Jul 07 11:19:41.057773 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:19:41.057754 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jul 07 11:19:41.086320 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:19:41.086297 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fvnjk\" (UniqueName: \"kubernetes.io/projected/9c339127-76e9-4527-8b21-bac4de7c933a-kube-api-access-fvnjk\") pod \"maas-keycloak-0\" (UID: \"9c339127-76e9-4527-8b21-bac4de7c933a\") " pod="keycloak-system/maas-keycloak-0" Jul 07 11:19:41.187573 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:19:41.187551 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fvnjk\" (UniqueName: \"kubernetes.io/projected/9c339127-76e9-4527-8b21-bac4de7c933a-kube-api-access-fvnjk\") pod \"maas-keycloak-0\" (UID: \"9c339127-76e9-4527-8b21-bac4de7c933a\") " pod="keycloak-system/maas-keycloak-0" Jul 07 11:19:41.197017 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:19:41.196993 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fvnjk\" (UniqueName: \"kubernetes.io/projected/9c339127-76e9-4527-8b21-bac4de7c933a-kube-api-access-fvnjk\") pod \"maas-keycloak-0\" (UID: \"9c339127-76e9-4527-8b21-bac4de7c933a\") " pod="keycloak-system/maas-keycloak-0" Jul 07 11:19:41.353021 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:19:41.352967 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jul 07 11:19:41.479961 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:19:41.479936 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jul 07 11:19:41.482001 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:19:41.481972 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c339127_76e9_4527_8b21_bac4de7c933a.slice/crio-a6e29a4fe73b31095d0f9d475bd614044580d01cb1c590e1d070cc5a07050070 WatchSource:0}: Error finding container a6e29a4fe73b31095d0f9d475bd614044580d01cb1c590e1d070cc5a07050070: Status 404 returned error can't find the container with id a6e29a4fe73b31095d0f9d475bd614044580d01cb1c590e1d070cc5a07050070 Jul 07 11:19:42.345018 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:19:42.344974 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"9c339127-76e9-4527-8b21-bac4de7c933a","Type":"ContainerStarted","Data":"a6e29a4fe73b31095d0f9d475bd614044580d01cb1c590e1d070cc5a07050070"} Jul 07 11:19:47.375263 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:19:47.375221 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"9c339127-76e9-4527-8b21-bac4de7c933a","Type":"ContainerStarted","Data":"f822e7b46c2e0c588830a9eb84035b8741a04874a98b9870fe212487d3d7c7f2"} Jul 07 11:19:47.397909 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:19:47.397848 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="keycloak-system/maas-keycloak-0" podStartSLOduration=1.450417788 podStartE2EDuration="6.397823728s" totalImagesPullingTime="4.94740594s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:19:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:19:47.393956642 +0000 UTC m=+511.807825808" watchObservedRunningTime="2026-07-07 11:19:47.397823728 +0000 UTC m=+511.811692895" Jul 07 11:19:48.353773 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:19:48.353733 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="keycloak-system/maas-keycloak-0" Jul 07 11:19:48.355274 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:19:48.355226 2562 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="9c339127-76e9-4527-8b21-bac4de7c933a" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.29:9000/health/started\": dial tcp 10.134.0.29:9000: connect: connection refused" Jul 07 11:19:49.353432 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:19:49.353395 2562 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="9c339127-76e9-4527-8b21-bac4de7c933a" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.29:9000/health/started\": dial tcp 10.134.0.29:9000: connect: connection refused" Jul 07 11:19:50.354287 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:19:50.354227 2562 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="9c339127-76e9-4527-8b21-bac4de7c933a" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.29:9000/health/started\": dial tcp 10.134.0.29:9000: connect: connection refused" Jul 07 11:19:51.354124 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:19:51.354065 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="keycloak-system/maas-keycloak-0" Jul 07 11:19:51.355124 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:19:51.355093 2562 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="9c339127-76e9-4527-8b21-bac4de7c933a" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.29:9000/health/started\": dial tcp 10.134.0.29:9000: connect: connection refused" Jul 07 11:19:52.354033 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:19:52.353989 2562 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="9c339127-76e9-4527-8b21-bac4de7c933a" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.29:9000/health/started\": dial tcp 10.134.0.29:9000: connect: connection refused" Jul 07 11:19:53.353666 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:19:53.353619 2562 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="9c339127-76e9-4527-8b21-bac4de7c933a" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.29:9000/health/started\": dial tcp 10.134.0.29:9000: connect: connection refused" Jul 07 11:19:54.354042 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:19:54.353999 2562 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="9c339127-76e9-4527-8b21-bac4de7c933a" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.29:9000/health/started\": dial tcp 10.134.0.29:9000: connect: connection refused" Jul 07 11:19:55.354204 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:19:55.354165 2562 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="9c339127-76e9-4527-8b21-bac4de7c933a" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.29:9000/health/started\": dial tcp 10.134.0.29:9000: connect: connection refused" Jul 07 11:19:56.353302 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:19:56.353251 2562 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="9c339127-76e9-4527-8b21-bac4de7c933a" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.29:9000/health/started\": dial tcp 10.134.0.29:9000: connect: connection refused" Jul 07 11:19:57.353704 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:19:57.353655 2562 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="9c339127-76e9-4527-8b21-bac4de7c933a" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.29:9000/health/started\": dial tcp 10.134.0.29:9000: connect: connection refused" Jul 07 11:19:58.353622 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:19:58.353579 2562 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="9c339127-76e9-4527-8b21-bac4de7c933a" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.29:9000/health/started\": dial tcp 10.134.0.29:9000: connect: connection refused" Jul 07 11:19:59.354269 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:19:59.354222 2562 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="9c339127-76e9-4527-8b21-bac4de7c933a" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.29:9000/health/started\": dial tcp 10.134.0.29:9000: connect: connection refused" Jul 07 11:20:00.478850 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:20:00.478773 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="keycloak-system/maas-keycloak-0" Jul 07 11:20:00.493656 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:20:00.493609 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="keycloak-system/maas-keycloak-0" podUID="9c339127-76e9-4527-8b21-bac4de7c933a" containerName="keycloak" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 07 11:20:10.486186 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:20:10.486155 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="keycloak-system/maas-keycloak-0" Jul 07 11:20:11.255141 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:20:11.255098 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:20:11.794093 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:20:11.794057 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:20:12.185963 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:20:12.185934 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-59665b5f85-c8wdn"] Jul 07 11:20:12.188295 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:20:12.188278 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59665b5f85-c8wdn" Jul 07 11:20:12.193420 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:20:12.193399 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-6v8rx\"" Jul 07 11:20:12.193420 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:20:12.193415 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 07 11:20:12.200716 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:20:12.200691 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59665b5f85-c8wdn"] Jul 07 11:20:12.301159 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:20:12.301133 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/8fb3fbfd-2ea3-4c41-bb7b-f16081af0a5a-tls-cert\") pod \"authorino-59665b5f85-c8wdn\" (UID: \"8fb3fbfd-2ea3-4c41-bb7b-f16081af0a5a\") " pod="kuadrant-system/authorino-59665b5f85-c8wdn" Jul 07 11:20:12.301280 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:20:12.301163 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k2jfp\" (UniqueName: \"kubernetes.io/projected/8fb3fbfd-2ea3-4c41-bb7b-f16081af0a5a-kube-api-access-k2jfp\") pod \"authorino-59665b5f85-c8wdn\" (UID: \"8fb3fbfd-2ea3-4c41-bb7b-f16081af0a5a\") " pod="kuadrant-system/authorino-59665b5f85-c8wdn" Jul 07 11:20:12.402186 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:20:12.402158 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/8fb3fbfd-2ea3-4c41-bb7b-f16081af0a5a-tls-cert\") pod \"authorino-59665b5f85-c8wdn\" (UID: \"8fb3fbfd-2ea3-4c41-bb7b-f16081af0a5a\") " pod="kuadrant-system/authorino-59665b5f85-c8wdn" Jul 07 11:20:12.402289 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:20:12.402188 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k2jfp\" (UniqueName: \"kubernetes.io/projected/8fb3fbfd-2ea3-4c41-bb7b-f16081af0a5a-kube-api-access-k2jfp\") pod \"authorino-59665b5f85-c8wdn\" (UID: \"8fb3fbfd-2ea3-4c41-bb7b-f16081af0a5a\") " pod="kuadrant-system/authorino-59665b5f85-c8wdn" Jul 07 11:20:12.404477 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:20:12.404457 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/8fb3fbfd-2ea3-4c41-bb7b-f16081af0a5a-tls-cert\") pod \"authorino-59665b5f85-c8wdn\" (UID: \"8fb3fbfd-2ea3-4c41-bb7b-f16081af0a5a\") " pod="kuadrant-system/authorino-59665b5f85-c8wdn" Jul 07 11:20:12.409930 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:20:12.409895 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k2jfp\" (UniqueName: \"kubernetes.io/projected/8fb3fbfd-2ea3-4c41-bb7b-f16081af0a5a-kube-api-access-k2jfp\") pod \"authorino-59665b5f85-c8wdn\" (UID: \"8fb3fbfd-2ea3-4c41-bb7b-f16081af0a5a\") " pod="kuadrant-system/authorino-59665b5f85-c8wdn" Jul 07 11:20:12.497937 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:20:12.497868 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59665b5f85-c8wdn" Jul 07 11:20:12.627753 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:20:12.627717 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59665b5f85-c8wdn"] Jul 07 11:20:12.631066 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:20:12.631039 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8fb3fbfd_2ea3_4c41_bb7b_f16081af0a5a.slice/crio-ab1892b2d6bec0702d63f8c9aae88f693683e12ad61cdf520973bab9c1594bc5 WatchSource:0}: Error finding container ab1892b2d6bec0702d63f8c9aae88f693683e12ad61cdf520973bab9c1594bc5: Status 404 returned error can't find the container with id ab1892b2d6bec0702d63f8c9aae88f693683e12ad61cdf520973bab9c1594bc5 Jul 07 11:20:13.007343 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:20:13.007310 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:20:13.534882 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:20:13.534848 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59665b5f85-c8wdn" event={"ID":"8fb3fbfd-2ea3-4c41-bb7b-f16081af0a5a","Type":"ContainerStarted","Data":"22cb39f031b560797535fe70b38168b65e28b12a1694bd93f1766c7c2ab825c7"} Jul 07 11:20:13.534882 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:20:13.534887 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59665b5f85-c8wdn" event={"ID":"8fb3fbfd-2ea3-4c41-bb7b-f16081af0a5a","Type":"ContainerStarted","Data":"ab1892b2d6bec0702d63f8c9aae88f693683e12ad61cdf520973bab9c1594bc5"} Jul 07 11:20:13.557019 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:20:13.556970 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-59665b5f85-c8wdn" podStartSLOduration=1.149409225 podStartE2EDuration="1.55695508s" totalImagesPullingTime="407.545855ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:20:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:20:13.556352797 +0000 UTC m=+537.970221961" watchObservedRunningTime="2026-07-07 11:20:13.55695508 +0000 UTC m=+537.970824243" Jul 07 11:20:18.363592 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:20:18.363561 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:20:18.474778 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:20:18.474748 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:20:18.575224 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:20:18.575181 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:20:18.810145 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:20:18.810112 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:20:19.770516 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:20:19.770484 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:20:19.851767 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:20:19.851739 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:20:22.255984 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:20:22.255947 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:20:22.424014 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:20:22.423957 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:20:54.801588 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:20:54.801537 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:20:55.359031 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:20:55.358995 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:20:55.613897 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:20:55.613804 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:20:56.122772 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:20:56.122724 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:20:56.308299 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:20:56.308270 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:20:56.452088 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:20:56.452057 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sns5p"] Jul 07 11:20:56.501056 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:20:56.501018 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:21:07.152831 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:21:07.152794 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jul 07 11:21:07.153308 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:21:07.153050 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="keycloak-system/maas-keycloak-0" podUID="9c339127-76e9-4527-8b21-bac4de7c933a" containerName="keycloak" containerID="cri-o://f822e7b46c2e0c588830a9eb84035b8741a04874a98b9870fe212487d3d7c7f2" gracePeriod=30 Jul 07 11:21:08.800015 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:21:08.799979 2562 generic.go:356] "Generic (PLEG): container finished" podID="9c339127-76e9-4527-8b21-bac4de7c933a" containerID="f822e7b46c2e0c588830a9eb84035b8741a04874a98b9870fe212487d3d7c7f2" exitCode=143 Jul 07 11:21:08.800375 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:21:08.800050 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"9c339127-76e9-4527-8b21-bac4de7c933a","Type":"ContainerDied","Data":"f822e7b46c2e0c588830a9eb84035b8741a04874a98b9870fe212487d3d7c7f2"} Jul 07 11:21:09.206426 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:21:09.206399 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jul 07 11:21:09.272843 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:21:09.272811 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fvnjk\" (UniqueName: \"kubernetes.io/projected/9c339127-76e9-4527-8b21-bac4de7c933a-kube-api-access-fvnjk\") pod \"9c339127-76e9-4527-8b21-bac4de7c933a\" (UID: \"9c339127-76e9-4527-8b21-bac4de7c933a\") " Jul 07 11:21:09.275114 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:21:09.275079 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9c339127-76e9-4527-8b21-bac4de7c933a-kube-api-access-fvnjk" (OuterVolumeSpecName: "kube-api-access-fvnjk") pod "9c339127-76e9-4527-8b21-bac4de7c933a" (UID: "9c339127-76e9-4527-8b21-bac4de7c933a"). InnerVolumeSpecName "kube-api-access-fvnjk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 11:21:09.374478 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:21:09.374442 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fvnjk\" (UniqueName: \"kubernetes.io/projected/9c339127-76e9-4527-8b21-bac4de7c933a-kube-api-access-fvnjk\") on node \"ip-10-0-136-15.ec2.internal\" DevicePath \"\"" Jul 07 11:21:09.806433 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:21:09.806398 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"9c339127-76e9-4527-8b21-bac4de7c933a","Type":"ContainerDied","Data":"a6e29a4fe73b31095d0f9d475bd614044580d01cb1c590e1d070cc5a07050070"} Jul 07 11:21:09.806887 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:21:09.806459 2562 scope.go:117] "RemoveContainer" containerID="f822e7b46c2e0c588830a9eb84035b8741a04874a98b9870fe212487d3d7c7f2" Jul 07 11:21:09.806887 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:21:09.806408 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jul 07 11:21:09.829092 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:21:09.829058 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jul 07 11:21:09.833965 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:21:09.833934 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jul 07 11:21:09.863934 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:21:09.863892 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["keycloak-system/maas-keycloak-0"] Jul 07 11:21:09.864317 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:21:09.864302 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9c339127-76e9-4527-8b21-bac4de7c933a" containerName="keycloak" Jul 07 11:21:09.864358 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:21:09.864319 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="9c339127-76e9-4527-8b21-bac4de7c933a" containerName="keycloak" Jul 07 11:21:09.864428 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:21:09.864417 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="9c339127-76e9-4527-8b21-bac4de7c933a" containerName="keycloak" Jul 07 11:21:09.868669 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:21:09.868649 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jul 07 11:21:09.872507 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:21:09.872483 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"maas-keycloak-initial-admin\"" Jul 07 11:21:09.872647 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:21:09.872549 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"default-dockercfg-hndvg\"" Jul 07 11:21:09.872647 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:21:09.872561 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"kube-root-ca.crt\"" Jul 07 11:21:09.872647 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:21:09.872477 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"openshift-service-ca.crt\"" Jul 07 11:21:09.873150 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:21:09.873133 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"keycloak-test-realms\"" Jul 07 11:21:09.891242 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:21:09.891205 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jul 07 11:21:09.979337 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:21:09.979300 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jgg8n\" (UniqueName: \"kubernetes.io/projected/075bc09f-e8d3-4254-88dd-e8292082d4c0-kube-api-access-jgg8n\") pod \"maas-keycloak-0\" (UID: \"075bc09f-e8d3-4254-88dd-e8292082d4c0\") " pod="keycloak-system/maas-keycloak-0" Jul 07 11:21:09.979521 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:21:09.979362 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"test-realms\" (UniqueName: \"kubernetes.io/configmap/075bc09f-e8d3-4254-88dd-e8292082d4c0-test-realms\") pod \"maas-keycloak-0\" (UID: \"075bc09f-e8d3-4254-88dd-e8292082d4c0\") " pod="keycloak-system/maas-keycloak-0" Jul 07 11:21:10.080581 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:21:10.080485 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jgg8n\" (UniqueName: \"kubernetes.io/projected/075bc09f-e8d3-4254-88dd-e8292082d4c0-kube-api-access-jgg8n\") pod \"maas-keycloak-0\" (UID: \"075bc09f-e8d3-4254-88dd-e8292082d4c0\") " pod="keycloak-system/maas-keycloak-0" Jul 07 11:21:10.080581 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:21:10.080556 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"test-realms\" (UniqueName: \"kubernetes.io/configmap/075bc09f-e8d3-4254-88dd-e8292082d4c0-test-realms\") pod \"maas-keycloak-0\" (UID: \"075bc09f-e8d3-4254-88dd-e8292082d4c0\") " pod="keycloak-system/maas-keycloak-0" Jul 07 11:21:10.081254 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:21:10.081235 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"test-realms\" (UniqueName: \"kubernetes.io/configmap/075bc09f-e8d3-4254-88dd-e8292082d4c0-test-realms\") pod \"maas-keycloak-0\" (UID: \"075bc09f-e8d3-4254-88dd-e8292082d4c0\") " pod="keycloak-system/maas-keycloak-0" Jul 07 11:21:10.090736 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:21:10.090714 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jgg8n\" (UniqueName: \"kubernetes.io/projected/075bc09f-e8d3-4254-88dd-e8292082d4c0-kube-api-access-jgg8n\") pod \"maas-keycloak-0\" (UID: \"075bc09f-e8d3-4254-88dd-e8292082d4c0\") " pod="keycloak-system/maas-keycloak-0" Jul 07 11:21:10.178410 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:21:10.178364 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jul 07 11:21:10.314581 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:21:10.314552 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jul 07 11:21:10.317425 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:21:10.317381 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod075bc09f_e8d3_4254_88dd_e8292082d4c0.slice/crio-d057a71c8d3e2c1535bc730584a56e720dd511faf96e8d206b6dc7420952f28c WatchSource:0}: Error finding container d057a71c8d3e2c1535bc730584a56e720dd511faf96e8d206b6dc7420952f28c: Status 404 returned error can't find the container with id d057a71c8d3e2c1535bc730584a56e720dd511faf96e8d206b6dc7420952f28c Jul 07 11:21:10.815052 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:21:10.815013 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"075bc09f-e8d3-4254-88dd-e8292082d4c0","Type":"ContainerStarted","Data":"e039973abb40c108b8a99656a717c52499393044a14c9930ab6483533b171348"} Jul 07 11:21:10.815476 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:21:10.815073 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"075bc09f-e8d3-4254-88dd-e8292082d4c0","Type":"ContainerStarted","Data":"d057a71c8d3e2c1535bc730584a56e720dd511faf96e8d206b6dc7420952f28c"} Jul 07 11:21:10.838288 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:21:10.838232 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="keycloak-system/maas-keycloak-0" podStartSLOduration=1.52343848 podStartE2EDuration="1.838214985s" totalImagesPullingTime="314.776505ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:21:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:21:10.836362172 +0000 UTC m=+595.250231374" watchObservedRunningTime="2026-07-07 11:21:10.838214985 +0000 UTC m=+595.252084150" Jul 07 11:21:11.179436 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:21:11.179389 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="keycloak-system/maas-keycloak-0" Jul 07 11:21:11.180869 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:21:11.180837 2562 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="075bc09f-e8d3-4254-88dd-e8292082d4c0" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.31:9000/health/started\": dial tcp 10.134.0.31:9000: connect: connection refused" Jul 07 11:21:11.769299 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:21:11.769268 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9c339127-76e9-4527-8b21-bac4de7c933a" path="/var/lib/kubelet/pods/9c339127-76e9-4527-8b21-bac4de7c933a/volumes" Jul 07 11:21:12.179364 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:21:12.179308 2562 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="075bc09f-e8d3-4254-88dd-e8292082d4c0" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.31:9000/health/started\": dial tcp 10.134.0.31:9000: connect: connection refused" Jul 07 11:21:12.521413 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:21:12.521319 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:21:13.179587 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:21:13.179539 2562 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="075bc09f-e8d3-4254-88dd-e8292082d4c0" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.31:9000/health/started\": dial tcp 10.134.0.31:9000: connect: connection refused" Jul 07 11:21:14.178867 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:21:14.178813 2562 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="075bc09f-e8d3-4254-88dd-e8292082d4c0" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.31:9000/health/started\": dial tcp 10.134.0.31:9000: connect: connection refused" Jul 07 11:21:15.179842 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:21:15.179796 2562 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="075bc09f-e8d3-4254-88dd-e8292082d4c0" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.31:9000/health/started\": dial tcp 10.134.0.31:9000: connect: connection refused" Jul 07 11:21:16.035080 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:21:16.035047 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tzf62_d2340b4b-179a-4c0a-9879-13b49897c939/ovn-acl-logging/0.log" Jul 07 11:21:16.035730 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:21:16.035705 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tzf62_d2340b4b-179a-4c0a-9879-13b49897c939/ovn-acl-logging/0.log" Jul 07 11:21:16.179848 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:21:16.179795 2562 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="075bc09f-e8d3-4254-88dd-e8292082d4c0" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.31:9000/health/started\": dial tcp 10.134.0.31:9000: connect: connection refused" Jul 07 11:21:17.179375 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:21:17.179320 2562 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="075bc09f-e8d3-4254-88dd-e8292082d4c0" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.31:9000/health/started\": dial tcp 10.134.0.31:9000: connect: connection refused" Jul 07 11:21:18.179134 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:21:18.179081 2562 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="075bc09f-e8d3-4254-88dd-e8292082d4c0" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.31:9000/health/started\": dial tcp 10.134.0.31:9000: connect: connection refused" Jul 07 11:21:18.509472 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:21:18.509305 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:21:19.179386 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:21:19.179347 2562 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="075bc09f-e8d3-4254-88dd-e8292082d4c0" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.31:9000/health/started\": dial tcp 10.134.0.31:9000: connect: connection refused" Jul 07 11:21:20.179264 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:21:20.179217 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="keycloak-system/maas-keycloak-0" Jul 07 11:21:20.179607 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:21:20.179574 2562 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="075bc09f-e8d3-4254-88dd-e8292082d4c0" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.31:9000/health/started\": dial tcp 10.134.0.31:9000: connect: connection refused" Jul 07 11:21:21.178906 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:21:21.178844 2562 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="075bc09f-e8d3-4254-88dd-e8292082d4c0" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.31:9000/health/started\": dial tcp 10.134.0.31:9000: connect: connection refused" Jul 07 11:21:22.179073 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:21:22.179023 2562 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="075bc09f-e8d3-4254-88dd-e8292082d4c0" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.31:9000/health/started\": dial tcp 10.134.0.31:9000: connect: connection refused" Jul 07 11:21:23.179439 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:21:23.179388 2562 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="075bc09f-e8d3-4254-88dd-e8292082d4c0" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.31:9000/health/started\": dial tcp 10.134.0.31:9000: connect: connection refused" Jul 07 11:21:24.179190 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:21:24.179137 2562 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="075bc09f-e8d3-4254-88dd-e8292082d4c0" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.31:9000/health/started\": dial tcp 10.134.0.31:9000: connect: connection refused" Jul 07 11:21:25.306011 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:21:25.305970 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="keycloak-system/maas-keycloak-0" Jul 07 11:21:25.324415 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:21:25.324367 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="keycloak-system/maas-keycloak-0" podUID="075bc09f-e8d3-4254-88dd-e8292082d4c0" containerName="keycloak" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 07 11:21:35.312112 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:21:35.312058 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="keycloak-system/maas-keycloak-0" Jul 07 11:21:39.958139 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:21:39.958056 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:21:47.860364 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:21:47.860321 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59665b5f85-c8wdn"] Jul 07 11:21:47.860777 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:21:47.860506 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-59665b5f85-c8wdn" podUID="8fb3fbfd-2ea3-4c41-bb7b-f16081af0a5a" containerName="authorino" containerID="cri-o://22cb39f031b560797535fe70b38168b65e28b12a1694bd93f1766c7c2ab825c7" gracePeriod=30 Jul 07 11:21:48.042344 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:21:48.042315 2562 generic.go:356] "Generic (PLEG): container finished" podID="8fb3fbfd-2ea3-4c41-bb7b-f16081af0a5a" containerID="22cb39f031b560797535fe70b38168b65e28b12a1694bd93f1766c7c2ab825c7" exitCode=0 Jul 07 11:21:48.042486 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:21:48.042385 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59665b5f85-c8wdn" event={"ID":"8fb3fbfd-2ea3-4c41-bb7b-f16081af0a5a","Type":"ContainerDied","Data":"22cb39f031b560797535fe70b38168b65e28b12a1694bd93f1766c7c2ab825c7"} Jul 07 11:21:48.118834 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:21:48.118773 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59665b5f85-c8wdn" Jul 07 11:21:48.174051 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:21:48.174021 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/8fb3fbfd-2ea3-4c41-bb7b-f16081af0a5a-tls-cert\") pod \"8fb3fbfd-2ea3-4c41-bb7b-f16081af0a5a\" (UID: \"8fb3fbfd-2ea3-4c41-bb7b-f16081af0a5a\") " Jul 07 11:21:48.174216 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:21:48.174137 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k2jfp\" (UniqueName: \"kubernetes.io/projected/8fb3fbfd-2ea3-4c41-bb7b-f16081af0a5a-kube-api-access-k2jfp\") pod \"8fb3fbfd-2ea3-4c41-bb7b-f16081af0a5a\" (UID: \"8fb3fbfd-2ea3-4c41-bb7b-f16081af0a5a\") " Jul 07 11:21:48.176065 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:21:48.176035 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8fb3fbfd-2ea3-4c41-bb7b-f16081af0a5a-kube-api-access-k2jfp" (OuterVolumeSpecName: "kube-api-access-k2jfp") pod "8fb3fbfd-2ea3-4c41-bb7b-f16081af0a5a" (UID: "8fb3fbfd-2ea3-4c41-bb7b-f16081af0a5a"). InnerVolumeSpecName "kube-api-access-k2jfp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 11:21:48.184045 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:21:48.184019 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8fb3fbfd-2ea3-4c41-bb7b-f16081af0a5a-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "8fb3fbfd-2ea3-4c41-bb7b-f16081af0a5a" (UID: "8fb3fbfd-2ea3-4c41-bb7b-f16081af0a5a"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 11:21:48.275531 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:21:48.275502 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k2jfp\" (UniqueName: \"kubernetes.io/projected/8fb3fbfd-2ea3-4c41-bb7b-f16081af0a5a-kube-api-access-k2jfp\") on node \"ip-10-0-136-15.ec2.internal\" DevicePath \"\"" Jul 07 11:21:48.275531 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:21:48.275526 2562 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/8fb3fbfd-2ea3-4c41-bb7b-f16081af0a5a-tls-cert\") on node \"ip-10-0-136-15.ec2.internal\" DevicePath \"\"" Jul 07 11:21:48.515119 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:21:48.515085 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:21:48.607844 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:21:48.607812 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:21:49.047970 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:21:49.047939 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59665b5f85-c8wdn" Jul 07 11:21:49.048356 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:21:49.047947 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59665b5f85-c8wdn" event={"ID":"8fb3fbfd-2ea3-4c41-bb7b-f16081af0a5a","Type":"ContainerDied","Data":"ab1892b2d6bec0702d63f8c9aae88f693683e12ad61cdf520973bab9c1594bc5"} Jul 07 11:21:49.048356 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:21:49.048060 2562 scope.go:117] "RemoveContainer" containerID="22cb39f031b560797535fe70b38168b65e28b12a1694bd93f1766c7c2ab825c7" Jul 07 11:21:49.069278 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:21:49.069245 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59665b5f85-c8wdn"] Jul 07 11:21:49.072029 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:21:49.072009 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-59665b5f85-c8wdn"] Jul 07 11:21:49.767469 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:21:49.767432 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8fb3fbfd-2ea3-4c41-bb7b-f16081af0a5a" path="/var/lib/kubelet/pods/8fb3fbfd-2ea3-4c41-bb7b-f16081af0a5a/volumes" Jul 07 11:21:52.820691 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:21:52.820655 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:21:55.631850 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:21:55.631817 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ncx5l"] Jul 07 11:21:55.632242 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:21:55.632218 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8fb3fbfd-2ea3-4c41-bb7b-f16081af0a5a" containerName="authorino" Jul 07 11:21:55.632242 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:21:55.632231 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="8fb3fbfd-2ea3-4c41-bb7b-f16081af0a5a" containerName="authorino" Jul 07 11:21:55.632318 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:21:55.632297 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="8fb3fbfd-2ea3-4c41-bb7b-f16081af0a5a" containerName="authorino" Jul 07 11:21:55.636644 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:21:55.636625 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ncx5l" Jul 07 11:21:55.639344 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:21:55.639323 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 07 11:21:55.640315 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:21:55.640297 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 07 11:21:55.640422 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:21:55.640404 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-6wgkw\"" Jul 07 11:21:55.640520 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:21:55.640501 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Jul 07 11:21:55.651583 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:21:55.651557 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ncx5l"] Jul 07 11:21:55.673933 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:21:55.673895 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:21:55.744364 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:21:55.744329 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c21c6534-63ce-4930-8053-a59ae7cc12be-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-ncx5l\" (UID: \"c21c6534-63ce-4930-8053-a59ae7cc12be\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ncx5l" Jul 07 11:21:55.744547 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:21:55.744382 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c21c6534-63ce-4930-8053-a59ae7cc12be-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-ncx5l\" (UID: \"c21c6534-63ce-4930-8053-a59ae7cc12be\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ncx5l" Jul 07 11:21:55.744547 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:21:55.744446 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4h85p\" (UniqueName: \"kubernetes.io/projected/c21c6534-63ce-4930-8053-a59ae7cc12be-kube-api-access-4h85p\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-ncx5l\" (UID: \"c21c6534-63ce-4930-8053-a59ae7cc12be\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ncx5l" Jul 07 11:21:55.744547 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:21:55.744506 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c21c6534-63ce-4930-8053-a59ae7cc12be-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-ncx5l\" (UID: \"c21c6534-63ce-4930-8053-a59ae7cc12be\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ncx5l" Jul 07 11:21:55.744547 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:21:55.744527 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c21c6534-63ce-4930-8053-a59ae7cc12be-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-ncx5l\" (UID: \"c21c6534-63ce-4930-8053-a59ae7cc12be\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ncx5l" Jul 07 11:21:55.744731 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:21:55.744654 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c21c6534-63ce-4930-8053-a59ae7cc12be-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-ncx5l\" (UID: \"c21c6534-63ce-4930-8053-a59ae7cc12be\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ncx5l" Jul 07 11:21:55.845955 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:21:55.845895 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c21c6534-63ce-4930-8053-a59ae7cc12be-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-ncx5l\" (UID: \"c21c6534-63ce-4930-8053-a59ae7cc12be\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ncx5l" Jul 07 11:21:55.845955 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:21:55.845954 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4h85p\" (UniqueName: \"kubernetes.io/projected/c21c6534-63ce-4930-8053-a59ae7cc12be-kube-api-access-4h85p\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-ncx5l\" (UID: \"c21c6534-63ce-4930-8053-a59ae7cc12be\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ncx5l" Jul 07 11:21:55.846209 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:21:55.846000 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c21c6534-63ce-4930-8053-a59ae7cc12be-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-ncx5l\" (UID: \"c21c6534-63ce-4930-8053-a59ae7cc12be\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ncx5l" Jul 07 11:21:55.846209 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:21:55.846026 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c21c6534-63ce-4930-8053-a59ae7cc12be-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-ncx5l\" (UID: \"c21c6534-63ce-4930-8053-a59ae7cc12be\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ncx5l" Jul 07 11:21:55.846209 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:21:55.846107 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c21c6534-63ce-4930-8053-a59ae7cc12be-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-ncx5l\" (UID: \"c21c6534-63ce-4930-8053-a59ae7cc12be\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ncx5l" Jul 07 11:21:55.846209 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:21:55.846166 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c21c6534-63ce-4930-8053-a59ae7cc12be-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-ncx5l\" (UID: \"c21c6534-63ce-4930-8053-a59ae7cc12be\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ncx5l" Jul 07 11:21:55.846434 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:21:55.846412 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c21c6534-63ce-4930-8053-a59ae7cc12be-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-ncx5l\" (UID: \"c21c6534-63ce-4930-8053-a59ae7cc12be\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ncx5l" Jul 07 11:21:55.846529 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:21:55.846507 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c21c6534-63ce-4930-8053-a59ae7cc12be-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-ncx5l\" (UID: \"c21c6534-63ce-4930-8053-a59ae7cc12be\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ncx5l" Jul 07 11:21:55.846592 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:21:55.846558 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c21c6534-63ce-4930-8053-a59ae7cc12be-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-ncx5l\" (UID: \"c21c6534-63ce-4930-8053-a59ae7cc12be\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ncx5l" Jul 07 11:21:55.848297 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:21:55.848274 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c21c6534-63ce-4930-8053-a59ae7cc12be-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-ncx5l\" (UID: \"c21c6534-63ce-4930-8053-a59ae7cc12be\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ncx5l" Jul 07 11:21:55.848402 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:21:55.848387 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c21c6534-63ce-4930-8053-a59ae7cc12be-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-ncx5l\" (UID: \"c21c6534-63ce-4930-8053-a59ae7cc12be\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ncx5l" Jul 07 11:21:55.856422 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:21:55.856396 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4h85p\" (UniqueName: \"kubernetes.io/projected/c21c6534-63ce-4930-8053-a59ae7cc12be-kube-api-access-4h85p\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-ncx5l\" (UID: \"c21c6534-63ce-4930-8053-a59ae7cc12be\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ncx5l" Jul 07 11:21:55.946558 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:21:55.946480 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ncx5l" Jul 07 11:21:56.083077 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:21:56.083044 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sns5p"] Jul 07 11:21:56.088317 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:21:56.088274 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ncx5l"] Jul 07 11:21:56.090791 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:21:56.090763 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc21c6534_63ce_4930_8053_a59ae7cc12be.slice/crio-3d7f1feff95c4ecc2a52f0d8f6fa1d6afc20d5ad0e2b9c515abd51f64915d1eb WatchSource:0}: Error finding container 3d7f1feff95c4ecc2a52f0d8f6fa1d6afc20d5ad0e2b9c515abd51f64915d1eb: Status 404 returned error can't find the container with id 3d7f1feff95c4ecc2a52f0d8f6fa1d6afc20d5ad0e2b9c515abd51f64915d1eb Jul 07 11:21:57.090134 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:21:57.090095 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ncx5l" event={"ID":"c21c6534-63ce-4930-8053-a59ae7cc12be","Type":"ContainerStarted","Data":"3d7f1feff95c4ecc2a52f0d8f6fa1d6afc20d5ad0e2b9c515abd51f64915d1eb"} Jul 07 11:21:58.096363 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:21:58.096327 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ncx5l" event={"ID":"c21c6534-63ce-4930-8053-a59ae7cc12be","Type":"ContainerStarted","Data":"5dc1f75543c903b429ee04668b999819057c027ee8dba14830fc5ec8fe29c54e"} Jul 07 11:21:58.116799 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:21:58.116737 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ncx5l" podStartSLOduration=1.990310233 podStartE2EDuration="3.116719735s" totalImagesPullingTime="1.126409502s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:21:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:21:58.115420793 +0000 UTC m=+642.529289963" watchObservedRunningTime="2026-07-07 11:21:58.116719735 +0000 UTC m=+642.530588900" Jul 07 11:22:05.947637 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:22:05.947598 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ncx5l" Jul 07 11:22:05.948114 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:22:05.947661 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ncx5l" Jul 07 11:22:05.959657 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:22:05.959637 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ncx5l" Jul 07 11:22:06.145176 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:22:06.145149 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ncx5l" Jul 07 11:22:15.327233 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:22:15.327202 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-z5jvw"] Jul 07 11:22:15.330615 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:22:15.330598 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-z5jvw" Jul 07 11:22:15.333302 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:22:15.333282 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jul 07 11:22:15.348422 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:22:15.348398 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-z5jvw"] Jul 07 11:22:15.363457 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:22:15.363435 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:22:15.416401 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:22:15.416378 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1199bebe-3056-49d7-bb3d-1e5dcf5c728f-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-z5jvw\" (UID: \"1199bebe-3056-49d7-bb3d-1e5dcf5c728f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-z5jvw" Jul 07 11:22:15.416487 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:22:15.416413 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1199bebe-3056-49d7-bb3d-1e5dcf5c728f-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-z5jvw\" (UID: \"1199bebe-3056-49d7-bb3d-1e5dcf5c728f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-z5jvw" Jul 07 11:22:15.416487 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:22:15.416432 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1199bebe-3056-49d7-bb3d-1e5dcf5c728f-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-z5jvw\" (UID: \"1199bebe-3056-49d7-bb3d-1e5dcf5c728f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-z5jvw" Jul 07 11:22:15.416569 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:22:15.416510 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1199bebe-3056-49d7-bb3d-1e5dcf5c728f-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-z5jvw\" (UID: \"1199bebe-3056-49d7-bb3d-1e5dcf5c728f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-z5jvw" Jul 07 11:22:15.416569 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:22:15.416536 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mrvq4\" (UniqueName: \"kubernetes.io/projected/1199bebe-3056-49d7-bb3d-1e5dcf5c728f-kube-api-access-mrvq4\") pod \"e2e-distinct-simulated-kserve-59975d69f9-z5jvw\" (UID: \"1199bebe-3056-49d7-bb3d-1e5dcf5c728f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-z5jvw" Jul 07 11:22:15.416668 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:22:15.416586 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1199bebe-3056-49d7-bb3d-1e5dcf5c728f-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-z5jvw\" (UID: \"1199bebe-3056-49d7-bb3d-1e5dcf5c728f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-z5jvw" Jul 07 11:22:15.517229 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:22:15.517203 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1199bebe-3056-49d7-bb3d-1e5dcf5c728f-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-z5jvw\" (UID: \"1199bebe-3056-49d7-bb3d-1e5dcf5c728f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-z5jvw" Jul 07 11:22:15.517343 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:22:15.517259 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1199bebe-3056-49d7-bb3d-1e5dcf5c728f-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-z5jvw\" (UID: \"1199bebe-3056-49d7-bb3d-1e5dcf5c728f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-z5jvw" Jul 07 11:22:15.517343 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:22:15.517284 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1199bebe-3056-49d7-bb3d-1e5dcf5c728f-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-z5jvw\" (UID: \"1199bebe-3056-49d7-bb3d-1e5dcf5c728f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-z5jvw" Jul 07 11:22:15.517343 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:22:15.517302 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1199bebe-3056-49d7-bb3d-1e5dcf5c728f-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-z5jvw\" (UID: \"1199bebe-3056-49d7-bb3d-1e5dcf5c728f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-z5jvw" Jul 07 11:22:15.517343 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:22:15.517332 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1199bebe-3056-49d7-bb3d-1e5dcf5c728f-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-z5jvw\" (UID: \"1199bebe-3056-49d7-bb3d-1e5dcf5c728f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-z5jvw" Jul 07 11:22:15.517565 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:22:15.517347 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mrvq4\" (UniqueName: \"kubernetes.io/projected/1199bebe-3056-49d7-bb3d-1e5dcf5c728f-kube-api-access-mrvq4\") pod \"e2e-distinct-simulated-kserve-59975d69f9-z5jvw\" (UID: \"1199bebe-3056-49d7-bb3d-1e5dcf5c728f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-z5jvw" Jul 07 11:22:15.517622 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:22:15.517593 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1199bebe-3056-49d7-bb3d-1e5dcf5c728f-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-z5jvw\" (UID: \"1199bebe-3056-49d7-bb3d-1e5dcf5c728f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-z5jvw" Jul 07 11:22:15.517683 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:22:15.517668 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1199bebe-3056-49d7-bb3d-1e5dcf5c728f-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-z5jvw\" (UID: \"1199bebe-3056-49d7-bb3d-1e5dcf5c728f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-z5jvw" Jul 07 11:22:15.517734 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:22:15.517720 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1199bebe-3056-49d7-bb3d-1e5dcf5c728f-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-z5jvw\" (UID: \"1199bebe-3056-49d7-bb3d-1e5dcf5c728f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-z5jvw" Jul 07 11:22:15.519526 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:22:15.519498 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1199bebe-3056-49d7-bb3d-1e5dcf5c728f-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-z5jvw\" (UID: \"1199bebe-3056-49d7-bb3d-1e5dcf5c728f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-z5jvw" Jul 07 11:22:15.519868 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:22:15.519846 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1199bebe-3056-49d7-bb3d-1e5dcf5c728f-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-z5jvw\" (UID: \"1199bebe-3056-49d7-bb3d-1e5dcf5c728f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-z5jvw" Jul 07 11:22:15.526088 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:22:15.526067 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mrvq4\" (UniqueName: \"kubernetes.io/projected/1199bebe-3056-49d7-bb3d-1e5dcf5c728f-kube-api-access-mrvq4\") pod \"e2e-distinct-simulated-kserve-59975d69f9-z5jvw\" (UID: \"1199bebe-3056-49d7-bb3d-1e5dcf5c728f\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-z5jvw" Jul 07 11:22:15.640731 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:22:15.640704 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-z5jvw" Jul 07 11:22:15.773582 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:22:15.773561 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-z5jvw"] Jul 07 11:22:15.775441 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:22:15.775413 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1199bebe_3056_49d7_bb3d_1e5dcf5c728f.slice/crio-81ebc2df8842eb1ada336e966c29c46886016cede2eb998e9fea8db772a5ebb8 WatchSource:0}: Error finding container 81ebc2df8842eb1ada336e966c29c46886016cede2eb998e9fea8db772a5ebb8: Status 404 returned error can't find the container with id 81ebc2df8842eb1ada336e966c29c46886016cede2eb998e9fea8db772a5ebb8 Jul 07 11:22:16.048195 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:22:16.048170 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sns5p"] Jul 07 11:22:16.191788 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:22:16.191754 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-z5jvw" event={"ID":"1199bebe-3056-49d7-bb3d-1e5dcf5c728f","Type":"ContainerStarted","Data":"feef77eb2add6ac1b6d2b8517bf6f2beebec5713595285c1b51bb6900a8555c3"} Jul 07 11:22:16.191986 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:22:16.191800 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-z5jvw" event={"ID":"1199bebe-3056-49d7-bb3d-1e5dcf5c728f","Type":"ContainerStarted","Data":"81ebc2df8842eb1ada336e966c29c46886016cede2eb998e9fea8db772a5ebb8"} Jul 07 11:22:25.641758 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:22:25.641720 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-z5jvw" Jul 07 11:22:25.642248 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:22:25.641809 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-z5jvw" Jul 07 11:22:25.654215 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:22:25.654197 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-z5jvw" Jul 07 11:22:25.678797 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:22:25.678752 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-z5jvw" podStartSLOduration=10.481882709 podStartE2EDuration="10.678737532s" totalImagesPullingTime="196.854823ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:22:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:22:16.215455994 +0000 UTC m=+660.629325160" watchObservedRunningTime="2026-07-07 11:22:25.678737532 +0000 UTC m=+670.092606697" Jul 07 11:22:26.249059 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:22:26.249027 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-z5jvw" Jul 07 11:22:34.721349 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:22:34.721312 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:22:35.768141 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:22:35.768110 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sns5p"] Jul 07 11:22:46.066867 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:22:46.066828 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:22:47.649693 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:22:47.649658 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sns5p"] Jul 07 11:22:57.573160 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:22:57.573126 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:22:59.461022 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:22:59.460981 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sns5p"] Jul 07 11:23:06.336296 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:23:06.336263 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:23:08.762323 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:23:08.762248 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sns5p"] Jul 07 11:23:37.672995 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:23:37.672954 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:23:37.793537 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:23:37.793511 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:23:38.995001 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:23:38.994960 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:23:39.249897 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:23:39.249811 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:23:49.094510 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:23:49.094475 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:24:07.690127 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:24:07.690090 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sns5p"] Jul 07 11:24:18.154279 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:24:18.154246 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sns5p"] Jul 07 11:24:29.352765 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:24:29.352727 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sns5p"] Jul 07 11:24:37.874364 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:24:37.874327 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sns5p"] Jul 07 11:24:48.958558 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:24:48.958520 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sns5p"] Jul 07 11:24:50.767209 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:24:50.767170 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:24:52.368145 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:24:52.368089 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:25:14.440273 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:25:14.440231 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sns5p"] Jul 07 11:25:25.548174 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:25:25.548138 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sns5p"] Jul 07 11:25:26.016076 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:25:26.016045 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:25:26.412106 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:25:26.412073 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:25:26.511878 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:25:26.511846 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:25:32.353247 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:25:32.353212 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:25:32.487097 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:25:32.487058 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:25:32.702608 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:25:32.702579 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:25:32.805095 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:25:32.805063 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:25:37.646336 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:25:37.646297 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sns5p"] Jul 07 11:25:38.380601 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:25:38.380566 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:25:38.531252 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:25:38.531225 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:25:54.746268 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:25:54.746233 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sns5p"] Jul 07 11:26:04.761387 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:26:04.761351 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sns5p"] Jul 07 11:26:12.842252 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:26:12.842216 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sns5p"] Jul 07 11:26:16.067064 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:26:16.067041 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tzf62_d2340b4b-179a-4c0a-9879-13b49897c939/ovn-acl-logging/0.log" Jul 07 11:26:16.069107 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:26:16.069087 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tzf62_d2340b4b-179a-4c0a-9879-13b49897c939/ovn-acl-logging/0.log" Jul 07 11:26:21.943606 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:26:21.943571 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sns5p"] Jul 07 11:26:31.766582 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:26:31.766554 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sns5p"] Jul 07 11:26:40.551401 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:26:40.551368 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sns5p"] Jul 07 11:26:51.843336 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:26:51.843302 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sns5p"] Jul 07 11:26:58.859814 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:26:58.859776 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sns5p"] Jul 07 11:27:05.874838 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:27:05.874792 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:27:06.247639 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:27:06.247559 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sns5p"] Jul 07 11:27:14.837112 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:27:14.837072 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sns5p"] Jul 07 11:27:27.652470 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:27:27.652430 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sns5p"] Jul 07 11:27:36.048262 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:27:36.048226 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sns5p"] Jul 07 11:28:00.653236 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:28:00.653150 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sns5p"] Jul 07 11:28:12.644808 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:28:12.644774 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sns5p"] Jul 07 11:28:28.743187 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:28:28.743147 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sns5p"] Jul 07 11:28:39.251318 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:28:39.251279 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sns5p"] Jul 07 11:28:54.964117 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:28:54.964080 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sns5p"] Jul 07 11:28:58.925676 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:28:58.925632 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:29:39.259775 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:29:39.259741 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:29:39.320357 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:29:39.320314 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:30:19.796373 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:30:19.796336 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:31:00.184451 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:31:00.184416 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sns5p"] Jul 07 11:31:09.473793 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:31:09.473760 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sns5p"] Jul 07 11:31:16.104766 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:31:16.104734 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tzf62_d2340b4b-179a-4c0a-9879-13b49897c939/ovn-acl-logging/0.log" Jul 07 11:31:16.107522 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:31:16.107503 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tzf62_d2340b4b-179a-4c0a-9879-13b49897c939/ovn-acl-logging/0.log" Jul 07 11:31:21.659044 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:31:21.659010 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sns5p"] Jul 07 11:31:30.564345 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:31:30.564300 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sns5p"] Jul 07 11:31:39.649612 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:31:39.649578 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sns5p"] Jul 07 11:31:48.343935 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:31:48.343878 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sns5p"] Jul 07 11:31:59.054122 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:31:59.054087 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sns5p"] Jul 07 11:32:10.352720 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:32:10.352627 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sns5p"] Jul 07 11:32:19.441923 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:32:19.441872 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:32:20.546826 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:32:20.546795 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sns5p"] Jul 07 11:32:30.035366 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:32:30.035329 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sns5p"] Jul 07 11:32:41.744965 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:32:41.744925 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sns5p"] Jul 07 11:32:49.253046 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:32:49.253011 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sns5p"] Jul 07 11:33:07.654554 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:33:07.654517 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sns5p"] Jul 07 11:33:16.567951 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:33:16.567898 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sns5p"] Jul 07 11:33:28.085786 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:33:28.085752 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sns5p"] Jul 07 11:33:39.345462 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:33:39.345378 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sns5p"] Jul 07 11:33:55.351518 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:33:55.351483 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sns5p"] Jul 07 11:34:02.089764 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:34:02.089728 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:34:04.845719 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:34:04.845679 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sns5p"] Jul 07 11:34:14.290871 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:34:14.290839 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sns5p"] Jul 07 11:34:24.543699 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:34:24.543661 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sns5p"] Jul 07 11:34:33.840538 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:34:33.840490 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sns5p"] Jul 07 11:34:42.642138 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:34:42.642104 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sns5p"] Jul 07 11:34:55.644017 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:34:55.643979 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sns5p"] Jul 07 11:35:04.433578 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:35:04.433543 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sns5p"] Jul 07 11:35:15.544595 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:35:15.544562 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sns5p"] Jul 07 11:35:28.046126 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:35:28.046094 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sns5p"] Jul 07 11:35:36.981369 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:35:36.981332 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sns5p"] Jul 07 11:35:48.090331 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:35:48.090299 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sns5p"] Jul 07 11:36:00.466351 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:36:00.466318 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sns5p"] Jul 07 11:36:12.854975 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:36:12.854938 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sns5p"] Jul 07 11:36:16.134729 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:36:16.134695 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tzf62_d2340b4b-179a-4c0a-9879-13b49897c939/ovn-acl-logging/0.log" Jul 07 11:36:16.139818 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:36:16.139796 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tzf62_d2340b4b-179a-4c0a-9879-13b49897c939/ovn-acl-logging/0.log" Jul 07 11:36:26.340710 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:36:26.340678 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sns5p"] Jul 07 11:36:31.549206 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:36:31.549128 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:36:31.634055 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:36:31.634018 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:36:37.285605 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:36:37.285569 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:36:37.387489 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:36:37.387434 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:36:37.522126 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:36:37.522091 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:36:37.658674 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:36:37.658641 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:36:38.761674 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:36:38.761624 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:36:38.869656 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:36:38.869617 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:36:39.007891 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:36:39.007838 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:36:39.226511 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:36:39.226480 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:36:39.709059 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:36:39.709017 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:36:39.809839 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:36:39.809807 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:36:41.262813 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:36:41.262776 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sns5p"] Jul 07 11:36:41.676046 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:36:41.676008 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:36:43.810946 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:36:43.810902 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:36:43.989467 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:36:43.989434 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:36:44.090896 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:36:44.090807 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:36:44.157158 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:36:44.157123 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:36:45.687431 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:36:45.687395 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:36:50.425682 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:36:50.425645 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:36:50.567633 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:36:50.567601 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:36:50.758532 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:36:50.758449 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:36:50.951241 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:36:50.951208 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:36:51.864733 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:36:51.864705 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:36:51.949566 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:36:51.949531 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:36:52.027392 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:36:52.027363 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:36:52.491999 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:36:52.491964 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:36:52.632507 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:36:52.632475 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:36:52.732493 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:36:52.732462 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:36:53.448570 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:36:53.448533 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sns5p"] Jul 07 11:36:54.015763 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:36:54.015734 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:36:57.029278 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:36:57.029243 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:36:57.373904 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:36:57.373873 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:37:03.411762 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:37:03.411726 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:37:03.510521 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:37:03.510490 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:37:09.912966 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:37:09.912924 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:37:10.014483 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:37:10.014444 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:37:10.132397 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:37:10.132361 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:37:10.853248 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:37:10.853216 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:37:10.953068 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:37:10.953040 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:37:11.399128 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:37:11.399095 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:37:11.527107 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:37:11.527076 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:37:11.629148 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:37:11.629119 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:37:11.786969 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:37:11.786870 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:37:12.126901 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:37:12.126868 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:37:12.227045 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:37:12.227011 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:37:14.149595 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:37:14.149561 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sns5p"] Jul 07 11:37:14.493284 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:37:14.493242 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:37:15.940688 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:37:15.940654 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:37:16.110039 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:37:16.109985 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:37:16.190497 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:37:16.190467 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:37:16.358741 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:37:16.358667 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:37:17.947373 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:37:17.947341 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:37:22.182163 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:37:22.182125 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:37:23.455494 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:37:23.455458 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:37:24.255566 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:37:24.255529 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:37:26.351250 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:37:26.351217 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sns5p"] Jul 07 11:37:30.592387 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:37:30.592351 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:37:30.825739 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:37:30.825705 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:37:30.966119 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:37:30.966087 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:37:31.453987 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:37:31.453955 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:37:57.424063 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:37:57.424024 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:38:00.288561 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:38:00.288521 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:38:00.402648 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:38:00.402613 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:38:04.586568 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:38:04.586532 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:38:04.730387 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:38:04.730356 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:38:05.025650 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:38:05.025600 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:38:05.181136 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:38:05.181107 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:38:20.957575 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:38:20.957540 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:38:27.061845 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:38:27.061804 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:38:27.171558 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:38:27.171517 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:38:27.272621 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:38:27.272585 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:38:37.817719 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:38:37.817684 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:38:38.644377 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:38:38.644335 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:38:44.693755 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:38:44.693714 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:38:44.805374 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:38:44.805335 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:38:44.943099 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:38:44.943066 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:38:45.085088 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:38:45.084990 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:38:46.215369 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:38:46.215332 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:38:52.104142 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:38:52.104105 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:38:52.216680 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:38:52.216639 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:38:52.320549 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:38:52.320506 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:38:52.424878 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:38:52.424843 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:38:53.272257 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:38:53.272217 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:38:53.368101 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:38:53.368070 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:38:55.035070 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:38:55.035042 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:38:55.346453 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:38:55.346364 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sns5p"] Jul 07 11:39:00.965714 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:39:00.965679 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:39:01.054406 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:39:01.054374 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:39:01.682605 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:39:01.682564 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:39:03.950850 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:39:03.950810 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sns5p"] Jul 07 11:39:07.750694 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:39:07.750655 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:39:07.888114 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:39:07.888077 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:39:08.083099 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:39:08.082999 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:39:08.396867 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:39:08.396837 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:39:08.489584 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:39:08.489551 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:39:11.709358 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:39:11.709315 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:39:11.795484 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:39:11.795447 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:39:11.897314 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:39:11.897283 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:39:12.028228 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:39:12.028142 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:39:12.389766 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:39:12.389739 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:39:12.491136 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:39:12.491105 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:39:16.184125 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:39:16.184087 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:39:16.865987 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:39:16.865953 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:39:16.957559 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:39:16.957528 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:39:17.338073 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:39:17.338039 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:39:18.048193 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:39:18.048155 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sns5p"] Jul 07 11:39:23.323143 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:39:23.323105 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:39:23.396221 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:39:23.396192 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:39:23.714160 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:39:23.714122 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:39:24.114246 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:39:24.114153 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:39:24.177900 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:39:24.177870 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:39:28.240516 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:39:28.240478 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sns5p"] Jul 07 11:39:30.245204 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:39:30.245164 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:39:30.347081 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:39:30.347052 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:39:30.511253 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:39:30.511175 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:39:30.836618 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:39:30.836536 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:39:30.978551 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:39:30.978518 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:39:31.085780 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:39:31.085746 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:39:31.180895 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:39:31.180862 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:39:36.929976 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:39:36.929910 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:39:36.993352 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:39:36.993314 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:39:37.095971 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:39:37.095937 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:39:37.279735 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:39:37.279636 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:39:43.045799 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:39:43.045708 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:39:43.107810 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:39:43.107773 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:39:49.462683 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:39:49.462647 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:39:49.806958 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:39:49.806842 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:39:50.102209 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:39:50.102123 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:39:52.904176 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:39:52.904136 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:39:52.995098 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:39:52.995063 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:39:53.100714 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:39:53.100679 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:39:53.209281 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:39:53.209184 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:39:53.523252 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:39:53.523156 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:39:53.624704 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:39:53.624662 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:39:57.411904 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:39:57.411862 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:39:57.848106 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:39:57.848015 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sns5p"] Jul 07 11:40:11.161701 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:40:11.161671 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:40:13.091545 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:40:13.091507 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:40:13.194729 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:40:13.194690 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:40:14.037942 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:40:14.037889 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sns5p"] Jul 07 11:40:14.342752 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:40:14.342679 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:40:14.443421 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:40:14.443396 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:40:37.453005 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:40:37.452971 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sns5p"] Jul 07 11:40:40.074219 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:40:40.074188 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:40:40.521144 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:40:40.521085 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:40:46.091257 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:40:46.091222 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:40:50.954513 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:40:50.954481 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sns5p"] Jul 07 11:40:51.144474 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:40:51.144439 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:41:16.166728 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:41:16.166696 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tzf62_d2340b4b-179a-4c0a-9879-13b49897c939/ovn-acl-logging/0.log" Jul 07 11:41:16.172697 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:41:16.172676 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tzf62_d2340b4b-179a-4c0a-9879-13b49897c939/ovn-acl-logging/0.log" Jul 07 11:41:27.839098 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:41:27.839065 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:41:29.327811 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:41:29.327743 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:41:31.150294 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:41:31.150259 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sns5p"] Jul 07 11:41:35.235377 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:41:35.235342 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:41:35.299623 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:41:35.299595 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:41:35.531973 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:41:35.531881 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:41:35.771775 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:41:35.771744 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:41:36.824085 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:41:36.824052 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:41:36.913364 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:41:36.913341 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:41:37.019950 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:41:37.019910 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:41:37.760952 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:41:37.760877 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:41:37.859923 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:41:37.859897 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:41:37.963877 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:41:37.963851 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:41:39.338094 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:41:39.338064 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:41:40.254273 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:41:40.254244 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sns5p"] Jul 07 11:41:45.666242 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:41:45.666183 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:41:45.763788 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:41:45.763751 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:41:46.376452 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:41:46.376420 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:41:51.690058 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:41:51.690023 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:41:52.633591 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:41:52.633558 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:41:52.757238 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:41:52.757205 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:41:52.859388 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:41:52.859353 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:41:54.151829 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:41:54.151792 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sns5p"] Jul 07 11:42:03.341156 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:42:03.341122 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:42:03.666604 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:42:03.666570 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:42:24.679429 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:42:24.679393 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:42:29.373320 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:42:29.373283 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:42:30.924822 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:42:30.924785 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:42:31.002828 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:42:31.002796 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:42:31.289661 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:42:31.289567 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:42:31.824674 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:42:31.824626 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:42:32.368060 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:42:32.368022 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:42:34.809681 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:42:34.809649 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:42:34.911605 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:42:34.911552 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:42:35.028314 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:42:35.028278 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:42:35.227118 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:42:35.227085 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:42:35.625059 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:42:35.625020 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:42:35.748380 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:42:35.748342 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:42:39.305538 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:42:39.305458 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:42:39.652653 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:42:39.652615 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:42:40.175168 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:42:40.175130 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:42:40.398492 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:42:40.398462 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:42:42.585742 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:42:42.585709 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sns5p"] Jul 07 11:42:42.909720 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:42:42.909681 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:42:43.096857 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:42:43.096825 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:42:43.302863 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:42:43.302774 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:42:43.544606 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:42:43.544567 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:42:43.933752 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:42:43.933719 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:42:44.033590 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:42:44.033561 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:42:47.373549 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:42:47.373513 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:42:49.960339 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:42:49.960299 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:42:57.683022 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:42:57.682988 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sns5p"] Jul 07 11:42:57.781211 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:42:57.781173 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:42:57.921630 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:42:57.921597 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:43:06.008714 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:43:06.008681 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:43:06.118605 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:43:06.118568 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:43:10.568864 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:43:10.568830 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-2507eb/test-model-2507eb-kserve-7c664c9587-vs4dk"] Jul 07 11:43:10.572530 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:43:10.572514 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-2507eb/test-model-2507eb-kserve-7c664c9587-vs4dk" Jul 07 11:43:10.577223 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:43:10.577187 2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-model-2507eb-kserve-7c664c9587-vs4dk\" is forbidden: User \"system:node:ip-10-0-136-15.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"ai-tenant-e2e-inf-b-2507eb\": no relationship found between node 'ip-10-0-136-15.ec2.internal' and this object" podUID="fcf79bfe-5a31-4a48-8aed-234cd81e608a" pod="ai-tenant-e2e-inf-b-2507eb/test-model-2507eb-kserve-7c664c9587-vs4dk" Jul 07 11:43:10.577344 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:43:10.577221 2562 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"test-model-2507eb-kserve-self-signed-certs\" is forbidden: User \"system:node:ip-10-0-136-15.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"ai-tenant-e2e-inf-b-2507eb\": no relationship found between node 'ip-10-0-136-15.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"ai-tenant-e2e-inf-b-2507eb\"/\"test-model-2507eb-kserve-self-signed-certs\"" type="*v1.Secret" Jul 07 11:43:10.577344 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:43:10.577289 2562 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"default-dockercfg-vx2pt\" is forbidden: User \"system:node:ip-10-0-136-15.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"ai-tenant-e2e-inf-b-2507eb\": no relationship found between node 'ip-10-0-136-15.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"ai-tenant-e2e-inf-b-2507eb\"/\"default-dockercfg-vx2pt\"" type="*v1.Secret" Jul 07 11:43:10.577525 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:43:10.577508 2562 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"openshift-service-ca.crt\" is forbidden: User \"system:node:ip-10-0-136-15.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"ai-tenant-e2e-inf-b-2507eb\": no relationship found between node 'ip-10-0-136-15.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"ai-tenant-e2e-inf-b-2507eb\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" Jul 07 11:43:10.578038 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:43:10.578015 2562 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"kube-root-ca.crt\" is forbidden: User \"system:node:ip-10-0-136-15.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"ai-tenant-e2e-inf-b-2507eb\": no relationship found between node 'ip-10-0-136-15.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"ai-tenant-e2e-inf-b-2507eb\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" Jul 07 11:43:10.607637 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:43:10.607606 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-2507eb/test-model-2507eb-kserve-7c664c9587-vs4dk"] Jul 07 11:43:10.637177 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:43:10.637147 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:43:10.647353 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:43:10.647329 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fcf79bfe-5a31-4a48-8aed-234cd81e608a-home\") pod \"test-model-2507eb-kserve-7c664c9587-vs4dk\" (UID: \"fcf79bfe-5a31-4a48-8aed-234cd81e608a\") " pod="ai-tenant-e2e-inf-b-2507eb/test-model-2507eb-kserve-7c664c9587-vs4dk" Jul 07 11:43:10.647444 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:43:10.647406 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fcf79bfe-5a31-4a48-8aed-234cd81e608a-tmp-dir\") pod \"test-model-2507eb-kserve-7c664c9587-vs4dk\" (UID: \"fcf79bfe-5a31-4a48-8aed-234cd81e608a\") " pod="ai-tenant-e2e-inf-b-2507eb/test-model-2507eb-kserve-7c664c9587-vs4dk" Jul 07 11:43:10.647444 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:43:10.647425 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fcf79bfe-5a31-4a48-8aed-234cd81e608a-tls-certs\") pod \"test-model-2507eb-kserve-7c664c9587-vs4dk\" (UID: \"fcf79bfe-5a31-4a48-8aed-234cd81e608a\") " pod="ai-tenant-e2e-inf-b-2507eb/test-model-2507eb-kserve-7c664c9587-vs4dk" Jul 07 11:43:10.647517 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:43:10.647442 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tdc6j\" (UniqueName: \"kubernetes.io/projected/fcf79bfe-5a31-4a48-8aed-234cd81e608a-kube-api-access-tdc6j\") pod \"test-model-2507eb-kserve-7c664c9587-vs4dk\" (UID: \"fcf79bfe-5a31-4a48-8aed-234cd81e608a\") " pod="ai-tenant-e2e-inf-b-2507eb/test-model-2507eb-kserve-7c664c9587-vs4dk" Jul 07 11:43:10.647600 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:43:10.647506 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fcf79bfe-5a31-4a48-8aed-234cd81e608a-dshm\") pod \"test-model-2507eb-kserve-7c664c9587-vs4dk\" (UID: \"fcf79bfe-5a31-4a48-8aed-234cd81e608a\") " pod="ai-tenant-e2e-inf-b-2507eb/test-model-2507eb-kserve-7c664c9587-vs4dk" Jul 07 11:43:10.647600 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:43:10.647556 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fcf79bfe-5a31-4a48-8aed-234cd81e608a-model-cache\") pod \"test-model-2507eb-kserve-7c664c9587-vs4dk\" (UID: \"fcf79bfe-5a31-4a48-8aed-234cd81e608a\") " pod="ai-tenant-e2e-inf-b-2507eb/test-model-2507eb-kserve-7c664c9587-vs4dk" Jul 07 11:43:10.748634 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:43:10.748536 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fcf79bfe-5a31-4a48-8aed-234cd81e608a-dshm\") pod \"test-model-2507eb-kserve-7c664c9587-vs4dk\" (UID: \"fcf79bfe-5a31-4a48-8aed-234cd81e608a\") " pod="ai-tenant-e2e-inf-b-2507eb/test-model-2507eb-kserve-7c664c9587-vs4dk" Jul 07 11:43:10.748634 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:43:10.748594 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fcf79bfe-5a31-4a48-8aed-234cd81e608a-model-cache\") pod \"test-model-2507eb-kserve-7c664c9587-vs4dk\" (UID: \"fcf79bfe-5a31-4a48-8aed-234cd81e608a\") " pod="ai-tenant-e2e-inf-b-2507eb/test-model-2507eb-kserve-7c664c9587-vs4dk" Jul 07 11:43:10.748634 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:43:10.748630 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fcf79bfe-5a31-4a48-8aed-234cd81e608a-home\") pod \"test-model-2507eb-kserve-7c664c9587-vs4dk\" (UID: \"fcf79bfe-5a31-4a48-8aed-234cd81e608a\") " pod="ai-tenant-e2e-inf-b-2507eb/test-model-2507eb-kserve-7c664c9587-vs4dk" Jul 07 11:43:10.748939 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:43:10.748778 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fcf79bfe-5a31-4a48-8aed-234cd81e608a-tmp-dir\") pod \"test-model-2507eb-kserve-7c664c9587-vs4dk\" (UID: \"fcf79bfe-5a31-4a48-8aed-234cd81e608a\") " pod="ai-tenant-e2e-inf-b-2507eb/test-model-2507eb-kserve-7c664c9587-vs4dk" Jul 07 11:43:10.748939 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:43:10.748817 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fcf79bfe-5a31-4a48-8aed-234cd81e608a-tls-certs\") pod \"test-model-2507eb-kserve-7c664c9587-vs4dk\" (UID: \"fcf79bfe-5a31-4a48-8aed-234cd81e608a\") " pod="ai-tenant-e2e-inf-b-2507eb/test-model-2507eb-kserve-7c664c9587-vs4dk" Jul 07 11:43:10.748939 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:43:10.748846 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tdc6j\" (UniqueName: \"kubernetes.io/projected/fcf79bfe-5a31-4a48-8aed-234cd81e608a-kube-api-access-tdc6j\") pod \"test-model-2507eb-kserve-7c664c9587-vs4dk\" (UID: \"fcf79bfe-5a31-4a48-8aed-234cd81e608a\") " pod="ai-tenant-e2e-inf-b-2507eb/test-model-2507eb-kserve-7c664c9587-vs4dk" Jul 07 11:43:10.749113 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:43:10.749063 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fcf79bfe-5a31-4a48-8aed-234cd81e608a-home\") pod \"test-model-2507eb-kserve-7c664c9587-vs4dk\" (UID: \"fcf79bfe-5a31-4a48-8aed-234cd81e608a\") " pod="ai-tenant-e2e-inf-b-2507eb/test-model-2507eb-kserve-7c664c9587-vs4dk" Jul 07 11:43:10.749169 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:43:10.749123 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fcf79bfe-5a31-4a48-8aed-234cd81e608a-model-cache\") pod \"test-model-2507eb-kserve-7c664c9587-vs4dk\" (UID: \"fcf79bfe-5a31-4a48-8aed-234cd81e608a\") " pod="ai-tenant-e2e-inf-b-2507eb/test-model-2507eb-kserve-7c664c9587-vs4dk" Jul 07 11:43:10.749169 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:43:10.749146 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fcf79bfe-5a31-4a48-8aed-234cd81e608a-tmp-dir\") pod \"test-model-2507eb-kserve-7c664c9587-vs4dk\" (UID: \"fcf79bfe-5a31-4a48-8aed-234cd81e608a\") " pod="ai-tenant-e2e-inf-b-2507eb/test-model-2507eb-kserve-7c664c9587-vs4dk" Jul 07 11:43:10.750818 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:43:10.750799 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fcf79bfe-5a31-4a48-8aed-234cd81e608a-dshm\") pod \"test-model-2507eb-kserve-7c664c9587-vs4dk\" (UID: \"fcf79bfe-5a31-4a48-8aed-234cd81e608a\") " pod="ai-tenant-e2e-inf-b-2507eb/test-model-2507eb-kserve-7c664c9587-vs4dk" Jul 07 11:43:11.397596 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:43:11.397563 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-2507eb\"/\"openshift-service-ca.crt\"" Jul 07 11:43:11.672462 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:43:11.672382 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-2507eb\"/\"kube-root-ca.crt\"" Jul 07 11:43:11.684461 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:43:11.684439 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tdc6j\" (UniqueName: \"kubernetes.io/projected/fcf79bfe-5a31-4a48-8aed-234cd81e608a-kube-api-access-tdc6j\") pod \"test-model-2507eb-kserve-7c664c9587-vs4dk\" (UID: \"fcf79bfe-5a31-4a48-8aed-234cd81e608a\") " pod="ai-tenant-e2e-inf-b-2507eb/test-model-2507eb-kserve-7c664c9587-vs4dk" Jul 07 11:43:11.749631 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:43:11.749602 2562 secret.go:189] Couldn't get secret ai-tenant-e2e-inf-b-2507eb/test-model-2507eb-kserve-self-signed-certs: failed to sync secret cache: timed out waiting for the condition Jul 07 11:43:11.749795 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:43:11.749691 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fcf79bfe-5a31-4a48-8aed-234cd81e608a-tls-certs podName:fcf79bfe-5a31-4a48-8aed-234cd81e608a nodeName:}" failed. No retries permitted until 2026-07-07 11:43:12.249670364 +0000 UTC m=+1916.663539527 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/fcf79bfe-5a31-4a48-8aed-234cd81e608a-tls-certs") pod "test-model-2507eb-kserve-7c664c9587-vs4dk" (UID: "fcf79bfe-5a31-4a48-8aed-234cd81e608a") : failed to sync secret cache: timed out waiting for the condition Jul 07 11:43:11.841556 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:43:11.841527 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-2507eb\"/\"default-dockercfg-vx2pt\"" Jul 07 11:43:11.970984 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:43:11.970884 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-2507eb\"/\"test-model-2507eb-kserve-self-signed-certs\"" Jul 07 11:43:12.264362 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:43:12.264274 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fcf79bfe-5a31-4a48-8aed-234cd81e608a-tls-certs\") pod \"test-model-2507eb-kserve-7c664c9587-vs4dk\" (UID: \"fcf79bfe-5a31-4a48-8aed-234cd81e608a\") " pod="ai-tenant-e2e-inf-b-2507eb/test-model-2507eb-kserve-7c664c9587-vs4dk" Jul 07 11:43:12.266729 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:43:12.266700 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fcf79bfe-5a31-4a48-8aed-234cd81e608a-tls-certs\") pod \"test-model-2507eb-kserve-7c664c9587-vs4dk\" (UID: \"fcf79bfe-5a31-4a48-8aed-234cd81e608a\") " pod="ai-tenant-e2e-inf-b-2507eb/test-model-2507eb-kserve-7c664c9587-vs4dk" Jul 07 11:43:12.381532 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:43:12.381500 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-2507eb/test-model-2507eb-kserve-7c664c9587-vs4dk" Jul 07 11:43:12.527341 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:43:12.527316 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-2507eb/test-model-2507eb-kserve-7c664c9587-vs4dk"] Jul 07 11:43:12.529060 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:43:12.529036 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfcf79bfe_5a31_4a48_8aed_234cd81e608a.slice/crio-bdee222b3daeca52c1995588e25133403e1e9111d71b3d2c726fdfc39b680e10 WatchSource:0}: Error finding container bdee222b3daeca52c1995588e25133403e1e9111d71b3d2c726fdfc39b680e10: Status 404 returned error can't find the container with id bdee222b3daeca52c1995588e25133403e1e9111d71b3d2c726fdfc39b680e10 Jul 07 11:43:13.197941 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:43:13.197896 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-2507eb/test-model-2507eb-kserve-7c664c9587-vs4dk" event={"ID":"fcf79bfe-5a31-4a48-8aed-234cd81e608a","Type":"ContainerStarted","Data":"5c6d84bddc10c834e12ee191876bc532c559b88536c6ace17f7f5086c53c870c"} Jul 07 11:43:13.197941 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:43:13.197947 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-2507eb/test-model-2507eb-kserve-7c664c9587-vs4dk" event={"ID":"fcf79bfe-5a31-4a48-8aed-234cd81e608a","Type":"ContainerStarted","Data":"bdee222b3daeca52c1995588e25133403e1e9111d71b3d2c726fdfc39b680e10"} Jul 07 11:43:13.225662 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:43:13.225608 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-2507eb/test-model-2507eb-kserve-7c664c9587-vs4dk" podStartSLOduration=3.225593242 podStartE2EDuration="3.225593242s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:43:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:43:13.223065057 +0000 UTC m=+1917.636934234" watchObservedRunningTime="2026-07-07 11:43:13.225593242 +0000 UTC m=+1917.639462406" Jul 07 11:43:17.350693 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:43:17.350659 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sns5p"] Jul 07 11:43:22.381972 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:43:22.381903 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-2507eb/test-model-2507eb-kserve-7c664c9587-vs4dk" Jul 07 11:43:22.382409 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:43:22.381992 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-2507eb/test-model-2507eb-kserve-7c664c9587-vs4dk" Jul 07 11:43:22.394642 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:43:22.394608 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-2507eb/test-model-2507eb-kserve-7c664c9587-vs4dk" Jul 07 11:43:23.259558 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:43:23.259522 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-2507eb/test-model-2507eb-kserve-7c664c9587-vs4dk" Jul 07 11:43:26.291817 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:43:26.291776 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:43:40.946931 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:43:40.946887 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:43:41.773023 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:43:41.772982 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-2507eb/test-model-2507eb-kserve-7c664c9587-vs4dk"] Jul 07 11:43:41.799004 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:43:41.798967 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:43:41.960320 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:43:41.960286 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:43:42.436536 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:43:42.436500 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:43:42.791870 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:43:42.791785 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:43:42.953826 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:43:42.953789 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:43:43.233883 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:43:43.233840 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:43:45.664661 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:43:45.664625 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sns5p"] Jul 07 11:43:47.457941 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:43:47.457898 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:43:56.792969 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:43:56.792901 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-2507eb/test-model-2507eb-kserve-7c664c9587-vs4dk" podUID="fcf79bfe-5a31-4a48-8aed-234cd81e608a" containerName="main" containerID="cri-o://5c6d84bddc10c834e12ee191876bc532c559b88536c6ace17f7f5086c53c870c" gracePeriod=45 Jul 07 11:43:57.045298 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:43:57.045245 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-2507eb/test-model-2507eb-kserve-7c664c9587-vs4dk" Jul 07 11:43:57.084405 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:43:57.082395 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tdc6j\" (UniqueName: \"kubernetes.io/projected/fcf79bfe-5a31-4a48-8aed-234cd81e608a-kube-api-access-tdc6j\") pod \"fcf79bfe-5a31-4a48-8aed-234cd81e608a\" (UID: \"fcf79bfe-5a31-4a48-8aed-234cd81e608a\") " Jul 07 11:43:57.084405 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:43:57.082449 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fcf79bfe-5a31-4a48-8aed-234cd81e608a-model-cache\") pod \"fcf79bfe-5a31-4a48-8aed-234cd81e608a\" (UID: \"fcf79bfe-5a31-4a48-8aed-234cd81e608a\") " Jul 07 11:43:57.084405 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:43:57.082498 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fcf79bfe-5a31-4a48-8aed-234cd81e608a-tmp-dir\") pod \"fcf79bfe-5a31-4a48-8aed-234cd81e608a\" (UID: \"fcf79bfe-5a31-4a48-8aed-234cd81e608a\") " Jul 07 11:43:57.084405 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:43:57.082531 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fcf79bfe-5a31-4a48-8aed-234cd81e608a-home\") pod \"fcf79bfe-5a31-4a48-8aed-234cd81e608a\" (UID: \"fcf79bfe-5a31-4a48-8aed-234cd81e608a\") " Jul 07 11:43:57.084405 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:43:57.082572 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fcf79bfe-5a31-4a48-8aed-234cd81e608a-tls-certs\") pod \"fcf79bfe-5a31-4a48-8aed-234cd81e608a\" (UID: \"fcf79bfe-5a31-4a48-8aed-234cd81e608a\") " Jul 07 11:43:57.084405 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:43:57.082607 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fcf79bfe-5a31-4a48-8aed-234cd81e608a-dshm\") pod \"fcf79bfe-5a31-4a48-8aed-234cd81e608a\" (UID: \"fcf79bfe-5a31-4a48-8aed-234cd81e608a\") " Jul 07 11:43:57.084405 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:43:57.083787 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fcf79bfe-5a31-4a48-8aed-234cd81e608a-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "fcf79bfe-5a31-4a48-8aed-234cd81e608a" (UID: "fcf79bfe-5a31-4a48-8aed-234cd81e608a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 11:43:57.084405 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:43:57.083985 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fcf79bfe-5a31-4a48-8aed-234cd81e608a-model-cache" (OuterVolumeSpecName: "model-cache") pod "fcf79bfe-5a31-4a48-8aed-234cd81e608a" (UID: "fcf79bfe-5a31-4a48-8aed-234cd81e608a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 11:43:57.084405 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:43:57.084154 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fcf79bfe-5a31-4a48-8aed-234cd81e608a-home" (OuterVolumeSpecName: "home") pod "fcf79bfe-5a31-4a48-8aed-234cd81e608a" (UID: "fcf79bfe-5a31-4a48-8aed-234cd81e608a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 11:43:57.088527 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:43:57.088480 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fcf79bfe-5a31-4a48-8aed-234cd81e608a-kube-api-access-tdc6j" (OuterVolumeSpecName: "kube-api-access-tdc6j") pod "fcf79bfe-5a31-4a48-8aed-234cd81e608a" (UID: "fcf79bfe-5a31-4a48-8aed-234cd81e608a"). InnerVolumeSpecName "kube-api-access-tdc6j". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 11:43:57.090989 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:43:57.090939 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fcf79bfe-5a31-4a48-8aed-234cd81e608a-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "fcf79bfe-5a31-4a48-8aed-234cd81e608a" (UID: "fcf79bfe-5a31-4a48-8aed-234cd81e608a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 11:43:57.091432 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:43:57.091325 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fcf79bfe-5a31-4a48-8aed-234cd81e608a-dshm" (OuterVolumeSpecName: "dshm") pod "fcf79bfe-5a31-4a48-8aed-234cd81e608a" (UID: "fcf79bfe-5a31-4a48-8aed-234cd81e608a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 11:43:57.184127 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:43:57.184094 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tdc6j\" (UniqueName: \"kubernetes.io/projected/fcf79bfe-5a31-4a48-8aed-234cd81e608a-kube-api-access-tdc6j\") on node \"ip-10-0-136-15.ec2.internal\" DevicePath \"\"" Jul 07 11:43:57.184127 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:43:57.184125 2562 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fcf79bfe-5a31-4a48-8aed-234cd81e608a-model-cache\") on node \"ip-10-0-136-15.ec2.internal\" DevicePath \"\"" Jul 07 11:43:57.184127 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:43:57.184134 2562 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fcf79bfe-5a31-4a48-8aed-234cd81e608a-tmp-dir\") on node \"ip-10-0-136-15.ec2.internal\" DevicePath \"\"" Jul 07 11:43:57.184300 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:43:57.184142 2562 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fcf79bfe-5a31-4a48-8aed-234cd81e608a-home\") on node \"ip-10-0-136-15.ec2.internal\" DevicePath \"\"" Jul 07 11:43:57.184300 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:43:57.184150 2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fcf79bfe-5a31-4a48-8aed-234cd81e608a-tls-certs\") on node \"ip-10-0-136-15.ec2.internal\" DevicePath \"\"" Jul 07 11:43:57.184300 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:43:57.184157 2562 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fcf79bfe-5a31-4a48-8aed-234cd81e608a-dshm\") on node \"ip-10-0-136-15.ec2.internal\" DevicePath \"\"" Jul 07 11:43:57.407454 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:43:57.407424 2562 generic.go:356] "Generic (PLEG): container finished" podID="fcf79bfe-5a31-4a48-8aed-234cd81e608a" containerID="5c6d84bddc10c834e12ee191876bc532c559b88536c6ace17f7f5086c53c870c" exitCode=0 Jul 07 11:43:57.407576 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:43:57.407485 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-2507eb/test-model-2507eb-kserve-7c664c9587-vs4dk" Jul 07 11:43:57.407576 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:43:57.407517 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-2507eb/test-model-2507eb-kserve-7c664c9587-vs4dk" event={"ID":"fcf79bfe-5a31-4a48-8aed-234cd81e608a","Type":"ContainerDied","Data":"5c6d84bddc10c834e12ee191876bc532c559b88536c6ace17f7f5086c53c870c"} Jul 07 11:43:57.407576 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:43:57.407570 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-2507eb/test-model-2507eb-kserve-7c664c9587-vs4dk" event={"ID":"fcf79bfe-5a31-4a48-8aed-234cd81e608a","Type":"ContainerDied","Data":"bdee222b3daeca52c1995588e25133403e1e9111d71b3d2c726fdfc39b680e10"} Jul 07 11:43:57.407685 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:43:57.407590 2562 scope.go:117] "RemoveContainer" containerID="5c6d84bddc10c834e12ee191876bc532c559b88536c6ace17f7f5086c53c870c" Jul 07 11:43:57.416479 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:43:57.416464 2562 scope.go:117] "RemoveContainer" containerID="5c6d84bddc10c834e12ee191876bc532c559b88536c6ace17f7f5086c53c870c" Jul 07 11:43:57.416722 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:43:57.416705 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5c6d84bddc10c834e12ee191876bc532c559b88536c6ace17f7f5086c53c870c\": container with ID starting with 5c6d84bddc10c834e12ee191876bc532c559b88536c6ace17f7f5086c53c870c not found: ID does not exist" containerID="5c6d84bddc10c834e12ee191876bc532c559b88536c6ace17f7f5086c53c870c" Jul 07 11:43:57.416766 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:43:57.416730 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5c6d84bddc10c834e12ee191876bc532c559b88536c6ace17f7f5086c53c870c"} err="failed to get container status \"5c6d84bddc10c834e12ee191876bc532c559b88536c6ace17f7f5086c53c870c\": rpc error: code = NotFound desc = could not find container \"5c6d84bddc10c834e12ee191876bc532c559b88536c6ace17f7f5086c53c870c\": container with ID starting with 5c6d84bddc10c834e12ee191876bc532c559b88536c6ace17f7f5086c53c870c not found: ID does not exist" Jul 07 11:43:57.428722 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:43:57.428700 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-2507eb/test-model-2507eb-kserve-7c664c9587-vs4dk"] Jul 07 11:43:57.433708 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:43:57.433686 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-2507eb/test-model-2507eb-kserve-7c664c9587-vs4dk"] Jul 07 11:43:57.767495 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:43:57.767435 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fcf79bfe-5a31-4a48-8aed-234cd81e608a" path="/var/lib/kubelet/pods/fcf79bfe-5a31-4a48-8aed-234cd81e608a/volumes" Jul 07 11:44:24.647046 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:44:24.647015 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:44:24.923735 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:44:24.923653 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:44:25.163553 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:44:25.163492 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:44:25.434215 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:44:25.434181 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:44:25.794155 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:44:25.794074 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:44:25.907732 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:44:25.907700 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:44:26.006098 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:44:26.006069 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:44:26.213784 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:44:26.213742 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:44:29.044761 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:44:29.044726 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sns5p"] Jul 07 11:44:32.240846 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:44:32.240811 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:44:32.346316 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:44:32.346282 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:44:32.553897 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:44:32.553806 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:44:32.751884 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:44:32.751851 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:44:33.093603 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:44:33.093574 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:44:33.405060 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:44:33.405029 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:44:33.710171 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:44:33.710086 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:44:36.514067 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:44:36.514028 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:44:36.645717 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:44:36.645676 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:44:36.773997 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:44:36.773885 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:44:37.018261 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:44:37.018225 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:44:37.595970 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:44:37.595937 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:44:37.724123 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:44:37.724091 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:44:40.935385 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:44:40.935352 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:44:41.414482 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:44:41.414440 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:44:41.730105 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:44:41.730017 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:44:42.082962 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:44:42.082856 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:44:42.344851 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:44:42.344768 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sns5p"] Jul 07 11:44:44.505191 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:44:44.505155 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:44:44.599668 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:44:44.599636 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:44:44.798158 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:44:44.798067 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:44:45.159739 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:44:45.159710 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:44:45.376605 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:44:45.376555 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:44:45.486194 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:44:45.486109 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:44:48.972348 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:44:48.972313 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:44:57.004138 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:44:57.004105 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:44:57.367133 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:44:57.367045 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:44:57.509200 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:44:57.509168 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:44:57.613016 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:44:57.612986 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:44:57.780071 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:44:57.780034 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:44:57.848644 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:44:57.848611 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sns5p"] Jul 07 11:45:00.133441 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:45:00.133403 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-29723745-pxrlh"] Jul 07 11:45:00.133941 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:45:00.133795 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fcf79bfe-5a31-4a48-8aed-234cd81e608a" containerName="main" Jul 07 11:45:00.133941 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:45:00.133808 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="fcf79bfe-5a31-4a48-8aed-234cd81e608a" containerName="main" Jul 07 11:45:00.133941 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:45:00.133868 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="fcf79bfe-5a31-4a48-8aed-234cd81e608a" containerName="main" Jul 07 11:45:00.137287 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:45:00.137267 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29723745-pxrlh" Jul 07 11:45:00.138497 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:45:00.138473 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-api-b-cd7fa7-29723745-cm5kv"] Jul 07 11:45:00.140174 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:45:00.140154 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-5dvr6\"" Jul 07 11:45:00.141653 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:45:00.141637 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-api-b-cd7fa7-29723745-cm5kv" Jul 07 11:45:00.146570 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:45:00.146550 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29723745-pxrlh"] Jul 07 11:45:00.152455 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:45:00.152429 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-api-b-cd7fa7-29723745-cm5kv"] Jul 07 11:45:00.267886 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:45:00.267854 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9wlr9\" (UniqueName: \"kubernetes.io/projected/bb90e11a-28dc-4bb8-a28c-d19d8272e6e1-kube-api-access-9wlr9\") pod \"maas-api-key-cleanup-29723745-pxrlh\" (UID: \"bb90e11a-28dc-4bb8-a28c-d19d8272e6e1\") " pod="opendatahub/maas-api-key-cleanup-29723745-pxrlh" Jul 07 11:45:00.268073 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:45:00.267951 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m64pk\" (UniqueName: \"kubernetes.io/projected/6c867627-7c02-445e-b859-fd03c6bff722-kube-api-access-m64pk\") pod \"maas-api-key-cleanup-e2e-api-b-cd7fa7-29723745-cm5kv\" (UID: \"6c867627-7c02-445e-b859-fd03c6bff722\") " pod="opendatahub/maas-api-key-cleanup-e2e-api-b-cd7fa7-29723745-cm5kv" Jul 07 11:45:00.369012 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:45:00.368970 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m64pk\" (UniqueName: \"kubernetes.io/projected/6c867627-7c02-445e-b859-fd03c6bff722-kube-api-access-m64pk\") pod \"maas-api-key-cleanup-e2e-api-b-cd7fa7-29723745-cm5kv\" (UID: \"6c867627-7c02-445e-b859-fd03c6bff722\") " pod="opendatahub/maas-api-key-cleanup-e2e-api-b-cd7fa7-29723745-cm5kv" Jul 07 11:45:00.369224 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:45:00.369061 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9wlr9\" (UniqueName: \"kubernetes.io/projected/bb90e11a-28dc-4bb8-a28c-d19d8272e6e1-kube-api-access-9wlr9\") pod \"maas-api-key-cleanup-29723745-pxrlh\" (UID: \"bb90e11a-28dc-4bb8-a28c-d19d8272e6e1\") " pod="opendatahub/maas-api-key-cleanup-29723745-pxrlh" Jul 07 11:45:00.378012 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:45:00.377981 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9wlr9\" (UniqueName: \"kubernetes.io/projected/bb90e11a-28dc-4bb8-a28c-d19d8272e6e1-kube-api-access-9wlr9\") pod \"maas-api-key-cleanup-29723745-pxrlh\" (UID: \"bb90e11a-28dc-4bb8-a28c-d19d8272e6e1\") " pod="opendatahub/maas-api-key-cleanup-29723745-pxrlh" Jul 07 11:45:00.378131 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:45:00.378050 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m64pk\" (UniqueName: \"kubernetes.io/projected/6c867627-7c02-445e-b859-fd03c6bff722-kube-api-access-m64pk\") pod \"maas-api-key-cleanup-e2e-api-b-cd7fa7-29723745-cm5kv\" (UID: \"6c867627-7c02-445e-b859-fd03c6bff722\") " pod="opendatahub/maas-api-key-cleanup-e2e-api-b-cd7fa7-29723745-cm5kv" Jul 07 11:45:00.450478 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:45:00.450405 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29723745-pxrlh" Jul 07 11:45:00.458218 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:45:00.458199 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-api-b-cd7fa7-29723745-cm5kv" Jul 07 11:45:00.616639 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:45:00.616605 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-api-b-cd7fa7-29723745-cm5kv"] Jul 07 11:45:00.619775 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:45:00.619748 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c867627_7c02_445e_b859_fd03c6bff722.slice/crio-1fb8d150d6de9948b43e49a21f599f1c0be33b3b1ee7791b7d580e7b40e5cddf WatchSource:0}: Error finding container 1fb8d150d6de9948b43e49a21f599f1c0be33b3b1ee7791b7d580e7b40e5cddf: Status 404 returned error can't find the container with id 1fb8d150d6de9948b43e49a21f599f1c0be33b3b1ee7791b7d580e7b40e5cddf Jul 07 11:45:00.621708 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:45:00.621693 2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 11:45:00.713348 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:45:00.713255 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-api-b-cd7fa7-29723745-cm5kv" event={"ID":"6c867627-7c02-445e-b859-fd03c6bff722","Type":"ContainerStarted","Data":"1fb8d150d6de9948b43e49a21f599f1c0be33b3b1ee7791b7d580e7b40e5cddf"} Jul 07 11:45:00.808342 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:45:00.808317 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29723745-pxrlh"] Jul 07 11:45:00.810411 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:45:00.810381 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb90e11a_28dc_4bb8_a28c_d19d8272e6e1.slice/crio-a39eec078197725cfb903467f4455b796c5efc3d63eed85bfec7ce0d0f1a5ea2 WatchSource:0}: Error finding container a39eec078197725cfb903467f4455b796c5efc3d63eed85bfec7ce0d0f1a5ea2: Status 404 returned error can't find the container with id a39eec078197725cfb903467f4455b796c5efc3d63eed85bfec7ce0d0f1a5ea2 Jul 07 11:45:01.720779 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:45:01.720731 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29723745-pxrlh" event={"ID":"bb90e11a-28dc-4bb8-a28c-d19d8272e6e1","Type":"ContainerStarted","Data":"a39eec078197725cfb903467f4455b796c5efc3d63eed85bfec7ce0d0f1a5ea2"} Jul 07 11:45:03.565926 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:45:03.565865 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:45:03.667741 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:45:03.667703 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:45:03.732137 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:45:03.732093 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-api-b-cd7fa7-29723745-cm5kv" event={"ID":"6c867627-7c02-445e-b859-fd03c6bff722","Type":"ContainerStarted","Data":"ce07740849e104b387923d94904290576b1b993f940193e7c4be456777762bed"} Jul 07 11:45:03.733650 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:45:03.733626 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29723745-pxrlh" event={"ID":"bb90e11a-28dc-4bb8-a28c-d19d8272e6e1","Type":"ContainerStarted","Data":"b21d3a2cd15d3e4c2f38b2e527e2fe1b87cd92d879a1f240780001d3bfe1a84f"} Jul 07 11:45:03.751338 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:45:03.751293 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-key-cleanup-e2e-api-b-cd7fa7-29723745-cm5kv" podStartSLOduration=1.56875206 podStartE2EDuration="3.75127677s" totalImagesPullingTime="2.18252471s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:45:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:45:03.749021125 +0000 UTC m=+2028.162890290" watchObservedRunningTime="2026-07-07 11:45:03.75127677 +0000 UTC m=+2028.165145913" Jul 07 11:45:03.769982 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:45:03.769903 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-key-cleanup-29723745-pxrlh" podStartSLOduration=1.772281237 podStartE2EDuration="3.76988286s" totalImagesPullingTime="1.997601623s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:45:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:45:03.76852943 +0000 UTC m=+2028.182398595" watchObservedRunningTime="2026-07-07 11:45:03.76988286 +0000 UTC m=+2028.183752026" Jul 07 11:45:03.831613 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:45:03.831578 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:45:04.050158 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:45:04.050125 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:45:04.355194 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:45:04.355162 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:45:04.609194 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:45:04.609106 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:45:04.724180 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:45:04.724147 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:45:04.740330 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:45:04.740292 2562 generic.go:356] "Generic (PLEG): container finished" podID="bb90e11a-28dc-4bb8-a28c-d19d8272e6e1" containerID="b21d3a2cd15d3e4c2f38b2e527e2fe1b87cd92d879a1f240780001d3bfe1a84f" exitCode=0 Jul 07 11:45:04.740515 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:45:04.740370 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29723745-pxrlh" event={"ID":"bb90e11a-28dc-4bb8-a28c-d19d8272e6e1","Type":"ContainerDied","Data":"b21d3a2cd15d3e4c2f38b2e527e2fe1b87cd92d879a1f240780001d3bfe1a84f"} Jul 07 11:45:04.741997 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:45:04.741973 2562 generic.go:356] "Generic (PLEG): container finished" podID="6c867627-7c02-445e-b859-fd03c6bff722" containerID="ce07740849e104b387923d94904290576b1b993f940193e7c4be456777762bed" exitCode=0 Jul 07 11:45:04.742110 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:45:04.742011 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-api-b-cd7fa7-29723745-cm5kv" event={"ID":"6c867627-7c02-445e-b859-fd03c6bff722","Type":"ContainerDied","Data":"ce07740849e104b387923d94904290576b1b993f940193e7c4be456777762bed"} Jul 07 11:45:04.802816 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:45:04.802778 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:45:04.985327 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:45:04.985289 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:45:04.998240 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:45:04.998210 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-api-b-cd7fa7-29723745-cm5kv"] Jul 07 11:45:05.902887 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:45:05.902865 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29723745-pxrlh" Jul 07 11:45:05.905841 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:45:05.905821 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-api-b-cd7fa7-29723745-cm5kv" Jul 07 11:45:06.027976 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:45:06.027852 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9wlr9\" (UniqueName: \"kubernetes.io/projected/bb90e11a-28dc-4bb8-a28c-d19d8272e6e1-kube-api-access-9wlr9\") pod \"bb90e11a-28dc-4bb8-a28c-d19d8272e6e1\" (UID: \"bb90e11a-28dc-4bb8-a28c-d19d8272e6e1\") " Jul 07 11:45:06.027976 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:45:06.027965 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m64pk\" (UniqueName: \"kubernetes.io/projected/6c867627-7c02-445e-b859-fd03c6bff722-kube-api-access-m64pk\") pod \"6c867627-7c02-445e-b859-fd03c6bff722\" (UID: \"6c867627-7c02-445e-b859-fd03c6bff722\") " Jul 07 11:45:06.030227 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:45:06.030193 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bb90e11a-28dc-4bb8-a28c-d19d8272e6e1-kube-api-access-9wlr9" (OuterVolumeSpecName: "kube-api-access-9wlr9") pod "bb90e11a-28dc-4bb8-a28c-d19d8272e6e1" (UID: "bb90e11a-28dc-4bb8-a28c-d19d8272e6e1"). InnerVolumeSpecName "kube-api-access-9wlr9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 11:45:06.030227 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:45:06.030213 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6c867627-7c02-445e-b859-fd03c6bff722-kube-api-access-m64pk" (OuterVolumeSpecName: "kube-api-access-m64pk") pod "6c867627-7c02-445e-b859-fd03c6bff722" (UID: "6c867627-7c02-445e-b859-fd03c6bff722"). InnerVolumeSpecName "kube-api-access-m64pk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 11:45:06.129649 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:45:06.129606 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9wlr9\" (UniqueName: \"kubernetes.io/projected/bb90e11a-28dc-4bb8-a28c-d19d8272e6e1-kube-api-access-9wlr9\") on node \"ip-10-0-136-15.ec2.internal\" DevicePath \"\"" Jul 07 11:45:06.129649 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:45:06.129643 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m64pk\" (UniqueName: \"kubernetes.io/projected/6c867627-7c02-445e-b859-fd03c6bff722-kube-api-access-m64pk\") on node \"ip-10-0-136-15.ec2.internal\" DevicePath \"\"" Jul 07 11:45:06.752990 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:45:06.752959 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-api-b-cd7fa7-29723745-cm5kv" Jul 07 11:45:06.752990 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:45:06.752973 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-api-b-cd7fa7-29723745-cm5kv" event={"ID":"6c867627-7c02-445e-b859-fd03c6bff722","Type":"ContainerDied","Data":"1fb8d150d6de9948b43e49a21f599f1c0be33b3b1ee7791b7d580e7b40e5cddf"} Jul 07 11:45:06.753219 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:45:06.753030 2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1fb8d150d6de9948b43e49a21f599f1c0be33b3b1ee7791b7d580e7b40e5cddf" Jul 07 11:45:06.754306 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:45:06.754282 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29723745-pxrlh" event={"ID":"bb90e11a-28dc-4bb8-a28c-d19d8272e6e1","Type":"ContainerDied","Data":"a39eec078197725cfb903467f4455b796c5efc3d63eed85bfec7ce0d0f1a5ea2"} Jul 07 11:45:06.754442 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:45:06.754326 2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a39eec078197725cfb903467f4455b796c5efc3d63eed85bfec7ce0d0f1a5ea2" Jul 07 11:45:06.754442 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:45:06.754294 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29723745-pxrlh" Jul 07 11:45:06.784610 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:45:06.784575 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-api-b-cd7fa7-29723745-cm5kv"] Jul 07 11:45:06.789693 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:45:06.789669 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-api-b-cd7fa7-29723745-cm5kv"] Jul 07 11:45:07.774369 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:45:07.774336 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6c867627-7c02-445e-b859-fd03c6bff722" path="/var/lib/kubelet/pods/6c867627-7c02-445e-b859-fd03c6bff722/volumes" Jul 07 11:45:09.259719 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:45:09.259682 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sns5p"] Jul 07 11:45:10.503728 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:45:10.503693 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:45:10.605691 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:45:10.605660 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:45:10.805047 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:45:10.804963 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:45:11.040277 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:45:11.040246 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:45:11.341188 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:45:11.341155 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:45:11.658074 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:45:11.658042 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:45:11.963163 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:45:11.963071 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:45:13.809724 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:45:13.809688 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:45:13.911680 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:45:13.911641 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:45:14.647113 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:45:14.647077 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:45:14.750396 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:45:14.750353 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:45:14.854810 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:45:14.854780 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:45:15.111205 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:45:15.111128 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:45:15.380685 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:45:15.380633 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:45:15.481859 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:45:15.481828 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:45:19.165784 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:45:19.165753 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:45:19.525565 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:45:19.525483 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:45:19.825910 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:45:19.825830 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:45:19.949093 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:45:19.949063 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sns5p"] Jul 07 11:45:20.005740 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:45:20.005705 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:45:22.725428 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:45:22.725394 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:45:22.774947 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:45:22.774902 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-shared-b-43ebe2-5d67f4d78d-d7hhc"] Jul 07 11:45:22.775300 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:45:22.775287 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb90e11a-28dc-4bb8-a28c-d19d8272e6e1" containerName="cleanup" Jul 07 11:45:22.775300 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:45:22.775301 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb90e11a-28dc-4bb8-a28c-d19d8272e6e1" containerName="cleanup" Jul 07 11:45:22.775402 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:45:22.775323 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6c867627-7c02-445e-b859-fd03c6bff722" containerName="cleanup" Jul 07 11:45:22.775402 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:45:22.775328 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="6c867627-7c02-445e-b859-fd03c6bff722" containerName="cleanup" Jul 07 11:45:22.775402 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:45:22.775394 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="bb90e11a-28dc-4bb8-a28c-d19d8272e6e1" containerName="cleanup" Jul 07 11:45:22.775510 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:45:22.775404 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="6c867627-7c02-445e-b859-fd03c6bff722" containerName="cleanup" Jul 07 11:45:22.778376 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:45:22.778358 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-43ebe2-5d67f4d78d-d7hhc" Jul 07 11:45:22.781231 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:45:22.781209 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-shared-b-43ebe2-serving-cert\"" Jul 07 11:45:22.782019 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:45:22.782005 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-5dvr6\"" Jul 07 11:45:22.788976 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:45:22.788955 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-43ebe2-5d67f4d78d-d7hhc"] Jul 07 11:45:22.828150 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:45:22.828119 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:45:22.880967 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:45:22.880937 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b94jq\" (UniqueName: \"kubernetes.io/projected/fa8dff7c-c144-4218-83bc-204ee7ea12e3-kube-api-access-b94jq\") pod \"maas-api-e2e-shared-b-43ebe2-5d67f4d78d-d7hhc\" (UID: \"fa8dff7c-c144-4218-83bc-204ee7ea12e3\") " pod="opendatahub/maas-api-e2e-shared-b-43ebe2-5d67f4d78d-d7hhc" Jul 07 11:45:22.880967 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:45:22.880970 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fa8dff7c-c144-4218-83bc-204ee7ea12e3-openshift-service-ca\") pod \"maas-api-e2e-shared-b-43ebe2-5d67f4d78d-d7hhc\" (UID: \"fa8dff7c-c144-4218-83bc-204ee7ea12e3\") " pod="opendatahub/maas-api-e2e-shared-b-43ebe2-5d67f4d78d-d7hhc" Jul 07 11:45:22.881138 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:45:22.880993 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fa8dff7c-c144-4218-83bc-204ee7ea12e3-maas-api-tls\") pod \"maas-api-e2e-shared-b-43ebe2-5d67f4d78d-d7hhc\" (UID: \"fa8dff7c-c144-4218-83bc-204ee7ea12e3\") " pod="opendatahub/maas-api-e2e-shared-b-43ebe2-5d67f4d78d-d7hhc" Jul 07 11:45:22.982448 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:45:22.982364 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b94jq\" (UniqueName: \"kubernetes.io/projected/fa8dff7c-c144-4218-83bc-204ee7ea12e3-kube-api-access-b94jq\") pod \"maas-api-e2e-shared-b-43ebe2-5d67f4d78d-d7hhc\" (UID: \"fa8dff7c-c144-4218-83bc-204ee7ea12e3\") " pod="opendatahub/maas-api-e2e-shared-b-43ebe2-5d67f4d78d-d7hhc" Jul 07 11:45:22.982448 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:45:22.982397 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fa8dff7c-c144-4218-83bc-204ee7ea12e3-openshift-service-ca\") pod \"maas-api-e2e-shared-b-43ebe2-5d67f4d78d-d7hhc\" (UID: \"fa8dff7c-c144-4218-83bc-204ee7ea12e3\") " pod="opendatahub/maas-api-e2e-shared-b-43ebe2-5d67f4d78d-d7hhc" Jul 07 11:45:22.982448 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:45:22.982423 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fa8dff7c-c144-4218-83bc-204ee7ea12e3-maas-api-tls\") pod \"maas-api-e2e-shared-b-43ebe2-5d67f4d78d-d7hhc\" (UID: \"fa8dff7c-c144-4218-83bc-204ee7ea12e3\") " pod="opendatahub/maas-api-e2e-shared-b-43ebe2-5d67f4d78d-d7hhc" Jul 07 11:45:22.982663 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:45:22.982538 2562 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-shared-b-43ebe2-serving-cert: secret "maas-api-e2e-shared-b-43ebe2-serving-cert" not found Jul 07 11:45:22.982663 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:45:22.982600 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fa8dff7c-c144-4218-83bc-204ee7ea12e3-maas-api-tls podName:fa8dff7c-c144-4218-83bc-204ee7ea12e3 nodeName:}" failed. No retries permitted until 2026-07-07 11:45:23.482584879 +0000 UTC m=+2047.896454023 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/fa8dff7c-c144-4218-83bc-204ee7ea12e3-maas-api-tls") pod "maas-api-e2e-shared-b-43ebe2-5d67f4d78d-d7hhc" (UID: "fa8dff7c-c144-4218-83bc-204ee7ea12e3") : secret "maas-api-e2e-shared-b-43ebe2-serving-cert" not found Jul 07 11:45:22.983077 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:45:22.983059 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fa8dff7c-c144-4218-83bc-204ee7ea12e3-openshift-service-ca\") pod \"maas-api-e2e-shared-b-43ebe2-5d67f4d78d-d7hhc\" (UID: \"fa8dff7c-c144-4218-83bc-204ee7ea12e3\") " pod="opendatahub/maas-api-e2e-shared-b-43ebe2-5d67f4d78d-d7hhc" Jul 07 11:45:23.005858 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:45:23.005834 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b94jq\" (UniqueName: \"kubernetes.io/projected/fa8dff7c-c144-4218-83bc-204ee7ea12e3-kube-api-access-b94jq\") pod \"maas-api-e2e-shared-b-43ebe2-5d67f4d78d-d7hhc\" (UID: \"fa8dff7c-c144-4218-83bc-204ee7ea12e3\") " pod="opendatahub/maas-api-e2e-shared-b-43ebe2-5d67f4d78d-d7hhc" Jul 07 11:45:23.185810 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:45:23.185768 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:45:23.393115 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:45:23.393087 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:45:23.487068 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:45:23.487032 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fa8dff7c-c144-4218-83bc-204ee7ea12e3-maas-api-tls\") pod \"maas-api-e2e-shared-b-43ebe2-5d67f4d78d-d7hhc\" (UID: \"fa8dff7c-c144-4218-83bc-204ee7ea12e3\") " pod="opendatahub/maas-api-e2e-shared-b-43ebe2-5d67f4d78d-d7hhc" Jul 07 11:45:23.489481 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:45:23.489457 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fa8dff7c-c144-4218-83bc-204ee7ea12e3-maas-api-tls\") pod \"maas-api-e2e-shared-b-43ebe2-5d67f4d78d-d7hhc\" (UID: \"fa8dff7c-c144-4218-83bc-204ee7ea12e3\") " pod="opendatahub/maas-api-e2e-shared-b-43ebe2-5d67f4d78d-d7hhc" Jul 07 11:45:23.495799 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:45:23.495776 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:45:23.689128 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:45:23.689026 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-43ebe2-5d67f4d78d-d7hhc" Jul 07 11:45:23.824026 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:45:23.823997 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-43ebe2-5d67f4d78d-d7hhc"] Jul 07 11:45:23.826205 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:45:23.826181 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa8dff7c_c144_4218_83bc_204ee7ea12e3.slice/crio-13c71b1966e02519fd1cbb437fd21c4e64140d5581fb4ee80c59f0facfa99ec8 WatchSource:0}: Error finding container 13c71b1966e02519fd1cbb437fd21c4e64140d5581fb4ee80c59f0facfa99ec8: Status 404 returned error can't find the container with id 13c71b1966e02519fd1cbb437fd21c4e64140d5581fb4ee80c59f0facfa99ec8 Jul 07 11:45:23.838464 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:45:23.838435 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-43ebe2-5d67f4d78d-d7hhc" event={"ID":"fa8dff7c-c144-4218-83bc-204ee7ea12e3","Type":"ContainerStarted","Data":"13c71b1966e02519fd1cbb437fd21c4e64140d5581fb4ee80c59f0facfa99ec8"} Jul 07 11:45:25.852112 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:45:25.852020 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-43ebe2-5d67f4d78d-d7hhc" event={"ID":"fa8dff7c-c144-4218-83bc-204ee7ea12e3","Type":"ContainerStarted","Data":"094efed1e97281203624a0cb76d612ce46face0a00ba5f07283bf799a1d20543"} Jul 07 11:45:25.852507 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:45:25.852158 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-shared-b-43ebe2-5d67f4d78d-d7hhc" Jul 07 11:45:25.870705 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:45:25.870658 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-shared-b-43ebe2-5d67f4d78d-d7hhc" podStartSLOduration=2.239027738 podStartE2EDuration="3.870645188s" totalImagesPullingTime="1.63161745s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:45:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:45:25.869644307 +0000 UTC m=+2050.283513495" watchObservedRunningTime="2026-07-07 11:45:25.870645188 +0000 UTC m=+2050.284514354" Jul 07 11:45:27.162629 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:45:27.162593 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:45:31.862992 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:45:31.862963 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-shared-b-43ebe2-5d67f4d78d-d7hhc" Jul 07 11:45:32.246179 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:45:32.246147 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:45:35.494442 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:45:35.494407 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sns5p"] Jul 07 11:45:36.473226 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:45:36.473194 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:45:36.587391 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:45:36.587359 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:45:45.030276 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:45:45.030193 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:45:45.130778 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:45:45.130745 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:45:57.427322 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:45:57.427283 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-b5d0aa/auth-test-model-b5d0aa-kserve-7bd8dcb8ff-mwn6r"] Jul 07 11:45:57.430842 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:45:57.430823 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b5d0aa/auth-test-model-b5d0aa-kserve-7bd8dcb8ff-mwn6r" Jul 07 11:45:57.434809 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:45:57.434778 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-b5d0aa\"/\"default-dockercfg-9ltmv\"" Jul 07 11:45:57.434809 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:45:57.434784 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-b5d0aa\"/\"openshift-service-ca.crt\"" Jul 07 11:45:57.435053 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:45:57.434810 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-b5d0aa\"/\"kube-root-ca.crt\"" Jul 07 11:45:57.435053 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:45:57.434848 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-b5d0aa\"/\"auth-test-model-b5d0aa-kserve-self-signed-certs\"" Jul 07 11:45:57.439697 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:45:57.439665 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-b5d0aa/auth-test-model-b5d0aa-kserve-7bd8dcb8ff-mwn6r"] Jul 07 11:45:57.530416 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:45:57.530384 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:45:57.600596 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:45:57.600551 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aa6dc051-3dcd-4835-b461-3c3abc110ec0-model-cache\") pod \"auth-test-model-b5d0aa-kserve-7bd8dcb8ff-mwn6r\" (UID: \"aa6dc051-3dcd-4835-b461-3c3abc110ec0\") " pod="ai-tenant-e2e-shared-a-b5d0aa/auth-test-model-b5d0aa-kserve-7bd8dcb8ff-mwn6r" Jul 07 11:45:57.600728 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:45:57.600610 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aa6dc051-3dcd-4835-b461-3c3abc110ec0-tls-certs\") pod \"auth-test-model-b5d0aa-kserve-7bd8dcb8ff-mwn6r\" (UID: \"aa6dc051-3dcd-4835-b461-3c3abc110ec0\") " pod="ai-tenant-e2e-shared-a-b5d0aa/auth-test-model-b5d0aa-kserve-7bd8dcb8ff-mwn6r" Jul 07 11:45:57.600728 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:45:57.600666 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dtwf2\" (UniqueName: \"kubernetes.io/projected/aa6dc051-3dcd-4835-b461-3c3abc110ec0-kube-api-access-dtwf2\") pod \"auth-test-model-b5d0aa-kserve-7bd8dcb8ff-mwn6r\" (UID: \"aa6dc051-3dcd-4835-b461-3c3abc110ec0\") " pod="ai-tenant-e2e-shared-a-b5d0aa/auth-test-model-b5d0aa-kserve-7bd8dcb8ff-mwn6r" Jul 07 11:45:57.600728 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:45:57.600707 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aa6dc051-3dcd-4835-b461-3c3abc110ec0-home\") pod \"auth-test-model-b5d0aa-kserve-7bd8dcb8ff-mwn6r\" (UID: \"aa6dc051-3dcd-4835-b461-3c3abc110ec0\") " pod="ai-tenant-e2e-shared-a-b5d0aa/auth-test-model-b5d0aa-kserve-7bd8dcb8ff-mwn6r" Jul 07 11:45:57.600728 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:45:57.600725 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aa6dc051-3dcd-4835-b461-3c3abc110ec0-dshm\") pod \"auth-test-model-b5d0aa-kserve-7bd8dcb8ff-mwn6r\" (UID: \"aa6dc051-3dcd-4835-b461-3c3abc110ec0\") " pod="ai-tenant-e2e-shared-a-b5d0aa/auth-test-model-b5d0aa-kserve-7bd8dcb8ff-mwn6r" Jul 07 11:45:57.600940 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:45:57.600768 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aa6dc051-3dcd-4835-b461-3c3abc110ec0-tmp-dir\") pod \"auth-test-model-b5d0aa-kserve-7bd8dcb8ff-mwn6r\" (UID: \"aa6dc051-3dcd-4835-b461-3c3abc110ec0\") " pod="ai-tenant-e2e-shared-a-b5d0aa/auth-test-model-b5d0aa-kserve-7bd8dcb8ff-mwn6r" Jul 07 11:45:57.701449 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:45:57.701383 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aa6dc051-3dcd-4835-b461-3c3abc110ec0-tmp-dir\") pod \"auth-test-model-b5d0aa-kserve-7bd8dcb8ff-mwn6r\" (UID: \"aa6dc051-3dcd-4835-b461-3c3abc110ec0\") " pod="ai-tenant-e2e-shared-a-b5d0aa/auth-test-model-b5d0aa-kserve-7bd8dcb8ff-mwn6r" Jul 07 11:45:57.701563 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:45:57.701472 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aa6dc051-3dcd-4835-b461-3c3abc110ec0-model-cache\") pod \"auth-test-model-b5d0aa-kserve-7bd8dcb8ff-mwn6r\" (UID: \"aa6dc051-3dcd-4835-b461-3c3abc110ec0\") " pod="ai-tenant-e2e-shared-a-b5d0aa/auth-test-model-b5d0aa-kserve-7bd8dcb8ff-mwn6r" Jul 07 11:45:57.701563 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:45:57.701514 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aa6dc051-3dcd-4835-b461-3c3abc110ec0-tls-certs\") pod \"auth-test-model-b5d0aa-kserve-7bd8dcb8ff-mwn6r\" (UID: \"aa6dc051-3dcd-4835-b461-3c3abc110ec0\") " pod="ai-tenant-e2e-shared-a-b5d0aa/auth-test-model-b5d0aa-kserve-7bd8dcb8ff-mwn6r" Jul 07 11:45:57.701563 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:45:57.701544 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dtwf2\" (UniqueName: \"kubernetes.io/projected/aa6dc051-3dcd-4835-b461-3c3abc110ec0-kube-api-access-dtwf2\") pod \"auth-test-model-b5d0aa-kserve-7bd8dcb8ff-mwn6r\" (UID: \"aa6dc051-3dcd-4835-b461-3c3abc110ec0\") " pod="ai-tenant-e2e-shared-a-b5d0aa/auth-test-model-b5d0aa-kserve-7bd8dcb8ff-mwn6r" Jul 07 11:45:57.701722 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:45:57.701593 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aa6dc051-3dcd-4835-b461-3c3abc110ec0-home\") pod \"auth-test-model-b5d0aa-kserve-7bd8dcb8ff-mwn6r\" (UID: \"aa6dc051-3dcd-4835-b461-3c3abc110ec0\") " pod="ai-tenant-e2e-shared-a-b5d0aa/auth-test-model-b5d0aa-kserve-7bd8dcb8ff-mwn6r" Jul 07 11:45:57.701722 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:45:57.701620 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aa6dc051-3dcd-4835-b461-3c3abc110ec0-dshm\") pod \"auth-test-model-b5d0aa-kserve-7bd8dcb8ff-mwn6r\" (UID: \"aa6dc051-3dcd-4835-b461-3c3abc110ec0\") " pod="ai-tenant-e2e-shared-a-b5d0aa/auth-test-model-b5d0aa-kserve-7bd8dcb8ff-mwn6r" Jul 07 11:45:57.701833 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:45:57.701750 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aa6dc051-3dcd-4835-b461-3c3abc110ec0-tmp-dir\") pod \"auth-test-model-b5d0aa-kserve-7bd8dcb8ff-mwn6r\" (UID: \"aa6dc051-3dcd-4835-b461-3c3abc110ec0\") " pod="ai-tenant-e2e-shared-a-b5d0aa/auth-test-model-b5d0aa-kserve-7bd8dcb8ff-mwn6r" Jul 07 11:45:57.701890 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:45:57.701836 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aa6dc051-3dcd-4835-b461-3c3abc110ec0-model-cache\") pod \"auth-test-model-b5d0aa-kserve-7bd8dcb8ff-mwn6r\" (UID: \"aa6dc051-3dcd-4835-b461-3c3abc110ec0\") " pod="ai-tenant-e2e-shared-a-b5d0aa/auth-test-model-b5d0aa-kserve-7bd8dcb8ff-mwn6r" Jul 07 11:45:57.701979 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:45:57.701893 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aa6dc051-3dcd-4835-b461-3c3abc110ec0-home\") pod \"auth-test-model-b5d0aa-kserve-7bd8dcb8ff-mwn6r\" (UID: \"aa6dc051-3dcd-4835-b461-3c3abc110ec0\") " pod="ai-tenant-e2e-shared-a-b5d0aa/auth-test-model-b5d0aa-kserve-7bd8dcb8ff-mwn6r" Jul 07 11:45:57.703837 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:45:57.703817 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aa6dc051-3dcd-4835-b461-3c3abc110ec0-dshm\") pod \"auth-test-model-b5d0aa-kserve-7bd8dcb8ff-mwn6r\" (UID: \"aa6dc051-3dcd-4835-b461-3c3abc110ec0\") " pod="ai-tenant-e2e-shared-a-b5d0aa/auth-test-model-b5d0aa-kserve-7bd8dcb8ff-mwn6r" Jul 07 11:45:57.704044 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:45:57.704028 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aa6dc051-3dcd-4835-b461-3c3abc110ec0-tls-certs\") pod \"auth-test-model-b5d0aa-kserve-7bd8dcb8ff-mwn6r\" (UID: \"aa6dc051-3dcd-4835-b461-3c3abc110ec0\") " pod="ai-tenant-e2e-shared-a-b5d0aa/auth-test-model-b5d0aa-kserve-7bd8dcb8ff-mwn6r" Jul 07 11:45:57.710810 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:45:57.710786 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dtwf2\" (UniqueName: \"kubernetes.io/projected/aa6dc051-3dcd-4835-b461-3c3abc110ec0-kube-api-access-dtwf2\") pod \"auth-test-model-b5d0aa-kserve-7bd8dcb8ff-mwn6r\" (UID: \"aa6dc051-3dcd-4835-b461-3c3abc110ec0\") " pod="ai-tenant-e2e-shared-a-b5d0aa/auth-test-model-b5d0aa-kserve-7bd8dcb8ff-mwn6r" Jul 07 11:45:57.741351 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:45:57.741332 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b5d0aa/auth-test-model-b5d0aa-kserve-7bd8dcb8ff-mwn6r" Jul 07 11:45:57.876658 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:45:57.876636 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-b5d0aa/auth-test-model-b5d0aa-kserve-7bd8dcb8ff-mwn6r"] Jul 07 11:45:57.878646 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:45:57.878615 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podaa6dc051_3dcd_4835_b461_3c3abc110ec0.slice/crio-aae7154bf8fd55d03561d2d681415c9e8b7501aa70554362c28dd2efaea98aa3 WatchSource:0}: Error finding container aae7154bf8fd55d03561d2d681415c9e8b7501aa70554362c28dd2efaea98aa3: Status 404 returned error can't find the container with id aae7154bf8fd55d03561d2d681415c9e8b7501aa70554362c28dd2efaea98aa3 Jul 07 11:45:58.007749 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:45:58.007674 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b5d0aa/auth-test-model-b5d0aa-kserve-7bd8dcb8ff-mwn6r" event={"ID":"aa6dc051-3dcd-4835-b461-3c3abc110ec0","Type":"ContainerStarted","Data":"1462b4d1bb9551d410181a4c7e25e31b339af5d33f10ae0f3e1fbfd2525d323b"} Jul 07 11:45:58.007749 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:45:58.007718 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b5d0aa/auth-test-model-b5d0aa-kserve-7bd8dcb8ff-mwn6r" event={"ID":"aa6dc051-3dcd-4835-b461-3c3abc110ec0","Type":"ContainerStarted","Data":"aae7154bf8fd55d03561d2d681415c9e8b7501aa70554362c28dd2efaea98aa3"} Jul 07 11:46:00.566105 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:46:00.566054 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-b5d0aa/auth-test-model-b5d0aa-kserve-7bd8dcb8ff-mwn6r" podStartSLOduration=3.566038624 podStartE2EDuration="3.566038624s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:45:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:45:58.026176083 +0000 UTC m=+2082.440045250" watchObservedRunningTime="2026-07-07 11:46:00.566038624 +0000 UTC m=+2084.979907789" Jul 07 11:46:00.566560 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:46:00.566546 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sns5p"] Jul 07 11:46:07.741699 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:46:07.741656 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-b5d0aa/auth-test-model-b5d0aa-kserve-7bd8dcb8ff-mwn6r" Jul 07 11:46:07.742236 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:46:07.741723 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-b5d0aa/auth-test-model-b5d0aa-kserve-7bd8dcb8ff-mwn6r" Jul 07 11:46:07.754232 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:46:07.754212 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-b5d0aa/auth-test-model-b5d0aa-kserve-7bd8dcb8ff-mwn6r" Jul 07 11:46:08.070270 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:46:08.070185 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-b5d0aa/auth-test-model-b5d0aa-kserve-7bd8dcb8ff-mwn6r" Jul 07 11:46:16.200903 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:46:16.200878 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tzf62_d2340b4b-179a-4c0a-9879-13b49897c939/ovn-acl-logging/0.log" Jul 07 11:46:16.206402 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:46:16.206382 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tzf62_d2340b4b-179a-4c0a-9879-13b49897c939/ovn-acl-logging/0.log" Jul 07 11:46:17.645822 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:46:17.645783 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:46:20.943663 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:46:20.943631 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sns5p"] Jul 07 11:46:30.218149 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:46:30.218111 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:46:30.329903 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:46:30.329872 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:46:38.651271 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:46:38.651226 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sns5p"] Jul 07 11:46:53.783619 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:46:53.783582 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sns5p"] Jul 07 11:47:09.068329 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:47:09.068240 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sns5p"] Jul 07 11:47:23.353521 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:47:23.353481 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sns5p"] Jul 07 11:47:39.459128 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:47:39.459094 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sns5p"] Jul 07 11:47:56.548760 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:47:56.548727 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sns5p"] Jul 07 11:48:09.728476 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:48:09.728441 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-b5d0aa/auth-test-model-b5d0aa-kserve-7bd8dcb8ff-mwn6r"] Jul 07 11:48:09.842967 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:48:09.842898 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:48:11.638430 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:48:11.638357 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:48:12.647036 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:48:12.647006 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sns5p"] Jul 07 11:48:15.156798 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:48:15.156721 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:48:24.748522 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:48:24.748463 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-b5d0aa/auth-test-model-b5d0aa-kserve-7bd8dcb8ff-mwn6r" podUID="aa6dc051-3dcd-4835-b461-3c3abc110ec0" containerName="main" containerID="cri-o://1462b4d1bb9551d410181a4c7e25e31b339af5d33f10ae0f3e1fbfd2525d323b" gracePeriod=45 Jul 07 11:48:25.001154 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:48:25.001095 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b5d0aa/auth-test-model-b5d0aa-kserve-7bd8dcb8ff-mwn6r" Jul 07 11:48:25.114410 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:48:25.114383 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aa6dc051-3dcd-4835-b461-3c3abc110ec0-tmp-dir\") pod \"aa6dc051-3dcd-4835-b461-3c3abc110ec0\" (UID: \"aa6dc051-3dcd-4835-b461-3c3abc110ec0\") " Jul 07 11:48:25.114592 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:48:25.114459 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aa6dc051-3dcd-4835-b461-3c3abc110ec0-tls-certs\") pod \"aa6dc051-3dcd-4835-b461-3c3abc110ec0\" (UID: \"aa6dc051-3dcd-4835-b461-3c3abc110ec0\") " Jul 07 11:48:25.114592 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:48:25.114546 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aa6dc051-3dcd-4835-b461-3c3abc110ec0-model-cache\") pod \"aa6dc051-3dcd-4835-b461-3c3abc110ec0\" (UID: \"aa6dc051-3dcd-4835-b461-3c3abc110ec0\") " Jul 07 11:48:25.114719 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:48:25.114593 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dtwf2\" (UniqueName: \"kubernetes.io/projected/aa6dc051-3dcd-4835-b461-3c3abc110ec0-kube-api-access-dtwf2\") pod \"aa6dc051-3dcd-4835-b461-3c3abc110ec0\" (UID: \"aa6dc051-3dcd-4835-b461-3c3abc110ec0\") " Jul 07 11:48:25.114719 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:48:25.114652 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aa6dc051-3dcd-4835-b461-3c3abc110ec0-dshm\") pod \"aa6dc051-3dcd-4835-b461-3c3abc110ec0\" (UID: \"aa6dc051-3dcd-4835-b461-3c3abc110ec0\") " Jul 07 11:48:25.114719 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:48:25.114680 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aa6dc051-3dcd-4835-b461-3c3abc110ec0-home\") pod \"aa6dc051-3dcd-4835-b461-3c3abc110ec0\" (UID: \"aa6dc051-3dcd-4835-b461-3c3abc110ec0\") " Jul 07 11:48:25.114719 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:48:25.114695 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa6dc051-3dcd-4835-b461-3c3abc110ec0-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "aa6dc051-3dcd-4835-b461-3c3abc110ec0" (UID: "aa6dc051-3dcd-4835-b461-3c3abc110ec0"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 11:48:25.114948 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:48:25.114802 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa6dc051-3dcd-4835-b461-3c3abc110ec0-model-cache" (OuterVolumeSpecName: "model-cache") pod "aa6dc051-3dcd-4835-b461-3c3abc110ec0" (UID: "aa6dc051-3dcd-4835-b461-3c3abc110ec0"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 11:48:25.115002 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:48:25.114988 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa6dc051-3dcd-4835-b461-3c3abc110ec0-home" (OuterVolumeSpecName: "home") pod "aa6dc051-3dcd-4835-b461-3c3abc110ec0" (UID: "aa6dc051-3dcd-4835-b461-3c3abc110ec0"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 11:48:25.115064 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:48:25.115053 2562 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aa6dc051-3dcd-4835-b461-3c3abc110ec0-model-cache\") on node \"ip-10-0-136-15.ec2.internal\" DevicePath \"\"" Jul 07 11:48:25.115119 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:48:25.115069 2562 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aa6dc051-3dcd-4835-b461-3c3abc110ec0-tmp-dir\") on node \"ip-10-0-136-15.ec2.internal\" DevicePath \"\"" Jul 07 11:48:25.116741 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:48:25.116714 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aa6dc051-3dcd-4835-b461-3c3abc110ec0-kube-api-access-dtwf2" (OuterVolumeSpecName: "kube-api-access-dtwf2") pod "aa6dc051-3dcd-4835-b461-3c3abc110ec0" (UID: "aa6dc051-3dcd-4835-b461-3c3abc110ec0"). InnerVolumeSpecName "kube-api-access-dtwf2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 11:48:25.116852 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:48:25.116738 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aa6dc051-3dcd-4835-b461-3c3abc110ec0-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "aa6dc051-3dcd-4835-b461-3c3abc110ec0" (UID: "aa6dc051-3dcd-4835-b461-3c3abc110ec0"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 11:48:25.116852 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:48:25.116749 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa6dc051-3dcd-4835-b461-3c3abc110ec0-dshm" (OuterVolumeSpecName: "dshm") pod "aa6dc051-3dcd-4835-b461-3c3abc110ec0" (UID: "aa6dc051-3dcd-4835-b461-3c3abc110ec0"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 11:48:25.215518 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:48:25.215493 2562 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aa6dc051-3dcd-4835-b461-3c3abc110ec0-dshm\") on node \"ip-10-0-136-15.ec2.internal\" DevicePath \"\"" Jul 07 11:48:25.215518 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:48:25.215515 2562 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aa6dc051-3dcd-4835-b461-3c3abc110ec0-home\") on node \"ip-10-0-136-15.ec2.internal\" DevicePath \"\"" Jul 07 11:48:25.215518 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:48:25.215524 2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aa6dc051-3dcd-4835-b461-3c3abc110ec0-tls-certs\") on node \"ip-10-0-136-15.ec2.internal\" DevicePath \"\"" Jul 07 11:48:25.215713 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:48:25.215533 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dtwf2\" (UniqueName: \"kubernetes.io/projected/aa6dc051-3dcd-4835-b461-3c3abc110ec0-kube-api-access-dtwf2\") on node \"ip-10-0-136-15.ec2.internal\" DevicePath \"\"" Jul 07 11:48:25.731812 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:48:25.731781 2562 generic.go:356] "Generic (PLEG): container finished" podID="aa6dc051-3dcd-4835-b461-3c3abc110ec0" containerID="1462b4d1bb9551d410181a4c7e25e31b339af5d33f10ae0f3e1fbfd2525d323b" exitCode=0 Jul 07 11:48:25.732052 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:48:25.731820 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b5d0aa/auth-test-model-b5d0aa-kserve-7bd8dcb8ff-mwn6r" Jul 07 11:48:25.732052 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:48:25.731869 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b5d0aa/auth-test-model-b5d0aa-kserve-7bd8dcb8ff-mwn6r" event={"ID":"aa6dc051-3dcd-4835-b461-3c3abc110ec0","Type":"ContainerDied","Data":"1462b4d1bb9551d410181a4c7e25e31b339af5d33f10ae0f3e1fbfd2525d323b"} Jul 07 11:48:25.732052 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:48:25.731941 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b5d0aa/auth-test-model-b5d0aa-kserve-7bd8dcb8ff-mwn6r" event={"ID":"aa6dc051-3dcd-4835-b461-3c3abc110ec0","Type":"ContainerDied","Data":"aae7154bf8fd55d03561d2d681415c9e8b7501aa70554362c28dd2efaea98aa3"} Jul 07 11:48:25.732052 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:48:25.731964 2562 scope.go:117] "RemoveContainer" containerID="1462b4d1bb9551d410181a4c7e25e31b339af5d33f10ae0f3e1fbfd2525d323b" Jul 07 11:48:25.741599 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:48:25.741578 2562 scope.go:117] "RemoveContainer" containerID="1462b4d1bb9551d410181a4c7e25e31b339af5d33f10ae0f3e1fbfd2525d323b" Jul 07 11:48:25.741874 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:48:25.741852 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1462b4d1bb9551d410181a4c7e25e31b339af5d33f10ae0f3e1fbfd2525d323b\": container with ID starting with 1462b4d1bb9551d410181a4c7e25e31b339af5d33f10ae0f3e1fbfd2525d323b not found: ID does not exist" containerID="1462b4d1bb9551d410181a4c7e25e31b339af5d33f10ae0f3e1fbfd2525d323b" Jul 07 11:48:25.741944 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:48:25.741885 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1462b4d1bb9551d410181a4c7e25e31b339af5d33f10ae0f3e1fbfd2525d323b"} err="failed to get container status \"1462b4d1bb9551d410181a4c7e25e31b339af5d33f10ae0f3e1fbfd2525d323b\": rpc error: code = NotFound desc = could not find container \"1462b4d1bb9551d410181a4c7e25e31b339af5d33f10ae0f3e1fbfd2525d323b\": container with ID starting with 1462b4d1bb9551d410181a4c7e25e31b339af5d33f10ae0f3e1fbfd2525d323b not found: ID does not exist" Jul 07 11:48:25.754180 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:48:25.754149 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-b5d0aa/auth-test-model-b5d0aa-kserve-7bd8dcb8ff-mwn6r"] Jul 07 11:48:25.758364 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:48:25.758334 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-b5d0aa/auth-test-model-b5d0aa-kserve-7bd8dcb8ff-mwn6r"] Jul 07 11:48:25.767601 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:48:25.767575 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aa6dc051-3dcd-4835-b461-3c3abc110ec0" path="/var/lib/kubelet/pods/aa6dc051-3dcd-4835-b461-3c3abc110ec0/volumes" Jul 07 11:48:27.854835 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:48:27.854799 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sns5p"] Jul 07 11:48:35.037224 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:48:35.037197 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:48:59.032375 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:48:59.032294 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-43ebe2/sub-test-model-43ebe2-kserve-848ff5cbdf-wjkqw"] Jul 07 11:48:59.032854 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:48:59.032688 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa6dc051-3dcd-4835-b461-3c3abc110ec0" containerName="main" Jul 07 11:48:59.032854 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:48:59.032699 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa6dc051-3dcd-4835-b461-3c3abc110ec0" containerName="main" Jul 07 11:48:59.032854 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:48:59.032784 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="aa6dc051-3dcd-4835-b461-3c3abc110ec0" containerName="main" Jul 07 11:48:59.035879 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:48:59.035863 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-43ebe2/sub-test-model-43ebe2-kserve-848ff5cbdf-wjkqw" Jul 07 11:48:59.039583 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:48:59.039564 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-43ebe2\"/\"default-dockercfg-8dc6c\"" Jul 07 11:48:59.039700 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:48:59.039601 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-43ebe2\"/\"openshift-service-ca.crt\"" Jul 07 11:48:59.039811 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:48:59.039797 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-43ebe2\"/\"kube-root-ca.crt\"" Jul 07 11:48:59.039880 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:48:59.039806 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-43ebe2\"/\"sub-test-model-43ebe2-kserve-self-signed-certs\"" Jul 07 11:48:59.047597 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:48:59.047577 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-43ebe2/sub-test-model-43ebe2-kserve-848ff5cbdf-wjkqw"] Jul 07 11:48:59.061847 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:48:59.061820 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:48:59.134660 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:48:59.134629 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/603cb437-c986-4f41-8153-17b4966d7e6d-tmp-dir\") pod \"sub-test-model-43ebe2-kserve-848ff5cbdf-wjkqw\" (UID: \"603cb437-c986-4f41-8153-17b4966d7e6d\") " pod="ai-tenant-e2e-shared-b-43ebe2/sub-test-model-43ebe2-kserve-848ff5cbdf-wjkqw" Jul 07 11:48:59.134806 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:48:59.134669 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/603cb437-c986-4f41-8153-17b4966d7e6d-home\") pod \"sub-test-model-43ebe2-kserve-848ff5cbdf-wjkqw\" (UID: \"603cb437-c986-4f41-8153-17b4966d7e6d\") " pod="ai-tenant-e2e-shared-b-43ebe2/sub-test-model-43ebe2-kserve-848ff5cbdf-wjkqw" Jul 07 11:48:59.134806 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:48:59.134712 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/603cb437-c986-4f41-8153-17b4966d7e6d-model-cache\") pod \"sub-test-model-43ebe2-kserve-848ff5cbdf-wjkqw\" (UID: \"603cb437-c986-4f41-8153-17b4966d7e6d\") " pod="ai-tenant-e2e-shared-b-43ebe2/sub-test-model-43ebe2-kserve-848ff5cbdf-wjkqw" Jul 07 11:48:59.134806 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:48:59.134751 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/603cb437-c986-4f41-8153-17b4966d7e6d-tls-certs\") pod \"sub-test-model-43ebe2-kserve-848ff5cbdf-wjkqw\" (UID: \"603cb437-c986-4f41-8153-17b4966d7e6d\") " pod="ai-tenant-e2e-shared-b-43ebe2/sub-test-model-43ebe2-kserve-848ff5cbdf-wjkqw" Jul 07 11:48:59.134910 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:48:59.134813 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/603cb437-c986-4f41-8153-17b4966d7e6d-dshm\") pod \"sub-test-model-43ebe2-kserve-848ff5cbdf-wjkqw\" (UID: \"603cb437-c986-4f41-8153-17b4966d7e6d\") " pod="ai-tenant-e2e-shared-b-43ebe2/sub-test-model-43ebe2-kserve-848ff5cbdf-wjkqw" Jul 07 11:48:59.134910 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:48:59.134862 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-97brs\" (UniqueName: \"kubernetes.io/projected/603cb437-c986-4f41-8153-17b4966d7e6d-kube-api-access-97brs\") pod \"sub-test-model-43ebe2-kserve-848ff5cbdf-wjkqw\" (UID: \"603cb437-c986-4f41-8153-17b4966d7e6d\") " pod="ai-tenant-e2e-shared-b-43ebe2/sub-test-model-43ebe2-kserve-848ff5cbdf-wjkqw" Jul 07 11:48:59.236260 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:48:59.236228 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/603cb437-c986-4f41-8153-17b4966d7e6d-home\") pod \"sub-test-model-43ebe2-kserve-848ff5cbdf-wjkqw\" (UID: \"603cb437-c986-4f41-8153-17b4966d7e6d\") " pod="ai-tenant-e2e-shared-b-43ebe2/sub-test-model-43ebe2-kserve-848ff5cbdf-wjkqw" Jul 07 11:48:59.236427 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:48:59.236280 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/603cb437-c986-4f41-8153-17b4966d7e6d-model-cache\") pod \"sub-test-model-43ebe2-kserve-848ff5cbdf-wjkqw\" (UID: \"603cb437-c986-4f41-8153-17b4966d7e6d\") " pod="ai-tenant-e2e-shared-b-43ebe2/sub-test-model-43ebe2-kserve-848ff5cbdf-wjkqw" Jul 07 11:48:59.236427 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:48:59.236309 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/603cb437-c986-4f41-8153-17b4966d7e6d-tls-certs\") pod \"sub-test-model-43ebe2-kserve-848ff5cbdf-wjkqw\" (UID: \"603cb437-c986-4f41-8153-17b4966d7e6d\") " pod="ai-tenant-e2e-shared-b-43ebe2/sub-test-model-43ebe2-kserve-848ff5cbdf-wjkqw" Jul 07 11:48:59.236427 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:48:59.236327 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/603cb437-c986-4f41-8153-17b4966d7e6d-dshm\") pod \"sub-test-model-43ebe2-kserve-848ff5cbdf-wjkqw\" (UID: \"603cb437-c986-4f41-8153-17b4966d7e6d\") " pod="ai-tenant-e2e-shared-b-43ebe2/sub-test-model-43ebe2-kserve-848ff5cbdf-wjkqw" Jul 07 11:48:59.236427 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:48:59.236354 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-97brs\" (UniqueName: \"kubernetes.io/projected/603cb437-c986-4f41-8153-17b4966d7e6d-kube-api-access-97brs\") pod \"sub-test-model-43ebe2-kserve-848ff5cbdf-wjkqw\" (UID: \"603cb437-c986-4f41-8153-17b4966d7e6d\") " pod="ai-tenant-e2e-shared-b-43ebe2/sub-test-model-43ebe2-kserve-848ff5cbdf-wjkqw" Jul 07 11:48:59.236667 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:48:59.236451 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/603cb437-c986-4f41-8153-17b4966d7e6d-tmp-dir\") pod \"sub-test-model-43ebe2-kserve-848ff5cbdf-wjkqw\" (UID: \"603cb437-c986-4f41-8153-17b4966d7e6d\") " pod="ai-tenant-e2e-shared-b-43ebe2/sub-test-model-43ebe2-kserve-848ff5cbdf-wjkqw" Jul 07 11:48:59.236727 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:48:59.236672 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/603cb437-c986-4f41-8153-17b4966d7e6d-home\") pod \"sub-test-model-43ebe2-kserve-848ff5cbdf-wjkqw\" (UID: \"603cb437-c986-4f41-8153-17b4966d7e6d\") " pod="ai-tenant-e2e-shared-b-43ebe2/sub-test-model-43ebe2-kserve-848ff5cbdf-wjkqw" Jul 07 11:48:59.236805 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:48:59.236782 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/603cb437-c986-4f41-8153-17b4966d7e6d-model-cache\") pod \"sub-test-model-43ebe2-kserve-848ff5cbdf-wjkqw\" (UID: \"603cb437-c986-4f41-8153-17b4966d7e6d\") " pod="ai-tenant-e2e-shared-b-43ebe2/sub-test-model-43ebe2-kserve-848ff5cbdf-wjkqw" Jul 07 11:48:59.236879 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:48:59.236864 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/603cb437-c986-4f41-8153-17b4966d7e6d-tmp-dir\") pod \"sub-test-model-43ebe2-kserve-848ff5cbdf-wjkqw\" (UID: \"603cb437-c986-4f41-8153-17b4966d7e6d\") " pod="ai-tenant-e2e-shared-b-43ebe2/sub-test-model-43ebe2-kserve-848ff5cbdf-wjkqw" Jul 07 11:48:59.238603 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:48:59.238574 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/603cb437-c986-4f41-8153-17b4966d7e6d-dshm\") pod \"sub-test-model-43ebe2-kserve-848ff5cbdf-wjkqw\" (UID: \"603cb437-c986-4f41-8153-17b4966d7e6d\") " pod="ai-tenant-e2e-shared-b-43ebe2/sub-test-model-43ebe2-kserve-848ff5cbdf-wjkqw" Jul 07 11:48:59.238895 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:48:59.238879 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/603cb437-c986-4f41-8153-17b4966d7e6d-tls-certs\") pod \"sub-test-model-43ebe2-kserve-848ff5cbdf-wjkqw\" (UID: \"603cb437-c986-4f41-8153-17b4966d7e6d\") " pod="ai-tenant-e2e-shared-b-43ebe2/sub-test-model-43ebe2-kserve-848ff5cbdf-wjkqw" Jul 07 11:48:59.246873 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:48:59.246854 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-97brs\" (UniqueName: \"kubernetes.io/projected/603cb437-c986-4f41-8153-17b4966d7e6d-kube-api-access-97brs\") pod \"sub-test-model-43ebe2-kserve-848ff5cbdf-wjkqw\" (UID: \"603cb437-c986-4f41-8153-17b4966d7e6d\") " pod="ai-tenant-e2e-shared-b-43ebe2/sub-test-model-43ebe2-kserve-848ff5cbdf-wjkqw" Jul 07 11:48:59.344523 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:48:59.344452 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-43ebe2/sub-test-model-43ebe2-kserve-848ff5cbdf-wjkqw" Jul 07 11:48:59.477805 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:48:59.477778 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-43ebe2/sub-test-model-43ebe2-kserve-848ff5cbdf-wjkqw"] Jul 07 11:48:59.479256 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:48:59.479221 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod603cb437_c986_4f41_8153_17b4966d7e6d.slice/crio-96db287440658a3d804bdf80acb045707456988fc8e8c22aab25b297fe8ecce0 WatchSource:0}: Error finding container 96db287440658a3d804bdf80acb045707456988fc8e8c22aab25b297fe8ecce0: Status 404 returned error can't find the container with id 96db287440658a3d804bdf80acb045707456988fc8e8c22aab25b297fe8ecce0 Jul 07 11:48:59.903030 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:48:59.902995 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-43ebe2/sub-test-model-43ebe2-kserve-848ff5cbdf-wjkqw" event={"ID":"603cb437-c986-4f41-8153-17b4966d7e6d","Type":"ContainerStarted","Data":"7c83bfb1461708d62df428717a12aacfa5cea193a3b658449538beec96540ab8"} Jul 07 11:48:59.903030 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:48:59.903033 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-43ebe2/sub-test-model-43ebe2-kserve-848ff5cbdf-wjkqw" event={"ID":"603cb437-c986-4f41-8153-17b4966d7e6d","Type":"ContainerStarted","Data":"96db287440658a3d804bdf80acb045707456988fc8e8c22aab25b297fe8ecce0"} Jul 07 11:48:59.928640 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:48:59.928588 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-43ebe2/sub-test-model-43ebe2-kserve-848ff5cbdf-wjkqw" podStartSLOduration=0.928563023 podStartE2EDuration="928.563023ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 11:48:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:48:59.927397993 +0000 UTC m=+2264.341267168" watchObservedRunningTime="2026-07-07 11:48:59.928563023 +0000 UTC m=+2264.342432188" Jul 07 11:49:03.418888 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:49:03.418855 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sns5p"] Jul 07 11:49:09.345102 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:49:09.345059 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-43ebe2/sub-test-model-43ebe2-kserve-848ff5cbdf-wjkqw" Jul 07 11:49:09.345533 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:49:09.345142 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-43ebe2/sub-test-model-43ebe2-kserve-848ff5cbdf-wjkqw" Jul 07 11:49:09.357736 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:49:09.357710 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-43ebe2/sub-test-model-43ebe2-kserve-848ff5cbdf-wjkqw" Jul 07 11:49:09.962810 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:49:09.962778 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-43ebe2/sub-test-model-43ebe2-kserve-848ff5cbdf-wjkqw" Jul 07 11:49:25.759715 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:49:25.759677 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sns5p"] Jul 07 11:49:42.554273 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:49:42.554232 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sns5p"] Jul 07 11:49:48.778956 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:49:48.778895 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:49:49.544765 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:49:49.544731 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-43ebe2/sub-test-model-43ebe2-kserve-848ff5cbdf-wjkqw"] Jul 07 11:49:49.568557 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:49:49.568532 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:49:52.452932 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:49:52.452891 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:50:00.348440 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:50:00.348407 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sns5p"] Jul 07 11:50:04.564888 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:50:04.564840 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-43ebe2/sub-test-model-43ebe2-kserve-848ff5cbdf-wjkqw" podUID="603cb437-c986-4f41-8153-17b4966d7e6d" containerName="main" containerID="cri-o://7c83bfb1461708d62df428717a12aacfa5cea193a3b658449538beec96540ab8" gracePeriod=45 Jul 07 11:50:04.815717 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:50:04.815661 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-43ebe2/sub-test-model-43ebe2-kserve-848ff5cbdf-wjkqw" Jul 07 11:50:04.918922 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:50:04.918891 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/603cb437-c986-4f41-8153-17b4966d7e6d-tls-certs\") pod \"603cb437-c986-4f41-8153-17b4966d7e6d\" (UID: \"603cb437-c986-4f41-8153-17b4966d7e6d\") " Jul 07 11:50:04.919066 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:50:04.918967 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-97brs\" (UniqueName: \"kubernetes.io/projected/603cb437-c986-4f41-8153-17b4966d7e6d-kube-api-access-97brs\") pod \"603cb437-c986-4f41-8153-17b4966d7e6d\" (UID: \"603cb437-c986-4f41-8153-17b4966d7e6d\") " Jul 07 11:50:04.919066 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:50:04.919014 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/603cb437-c986-4f41-8153-17b4966d7e6d-model-cache\") pod \"603cb437-c986-4f41-8153-17b4966d7e6d\" (UID: \"603cb437-c986-4f41-8153-17b4966d7e6d\") " Jul 07 11:50:04.919066 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:50:04.919049 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/603cb437-c986-4f41-8153-17b4966d7e6d-dshm\") pod \"603cb437-c986-4f41-8153-17b4966d7e6d\" (UID: \"603cb437-c986-4f41-8153-17b4966d7e6d\") " Jul 07 11:50:04.919236 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:50:04.919075 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/603cb437-c986-4f41-8153-17b4966d7e6d-tmp-dir\") pod \"603cb437-c986-4f41-8153-17b4966d7e6d\" (UID: \"603cb437-c986-4f41-8153-17b4966d7e6d\") " Jul 07 11:50:04.919236 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:50:04.919108 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/603cb437-c986-4f41-8153-17b4966d7e6d-home\") pod \"603cb437-c986-4f41-8153-17b4966d7e6d\" (UID: \"603cb437-c986-4f41-8153-17b4966d7e6d\") " Jul 07 11:50:04.919355 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:50:04.919269 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/603cb437-c986-4f41-8153-17b4966d7e6d-model-cache" (OuterVolumeSpecName: "model-cache") pod "603cb437-c986-4f41-8153-17b4966d7e6d" (UID: "603cb437-c986-4f41-8153-17b4966d7e6d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 11:50:04.919425 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:50:04.919373 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/603cb437-c986-4f41-8153-17b4966d7e6d-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "603cb437-c986-4f41-8153-17b4966d7e6d" (UID: "603cb437-c986-4f41-8153-17b4966d7e6d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 11:50:04.919482 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:50:04.919438 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/603cb437-c986-4f41-8153-17b4966d7e6d-home" (OuterVolumeSpecName: "home") pod "603cb437-c986-4f41-8153-17b4966d7e6d" (UID: "603cb437-c986-4f41-8153-17b4966d7e6d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 11:50:04.919535 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:50:04.919515 2562 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/603cb437-c986-4f41-8153-17b4966d7e6d-model-cache\") on node \"ip-10-0-136-15.ec2.internal\" DevicePath \"\"" Jul 07 11:50:04.921272 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:50:04.921237 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/603cb437-c986-4f41-8153-17b4966d7e6d-dshm" (OuterVolumeSpecName: "dshm") pod "603cb437-c986-4f41-8153-17b4966d7e6d" (UID: "603cb437-c986-4f41-8153-17b4966d7e6d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 11:50:04.921272 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:50:04.921245 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/603cb437-c986-4f41-8153-17b4966d7e6d-kube-api-access-97brs" (OuterVolumeSpecName: "kube-api-access-97brs") pod "603cb437-c986-4f41-8153-17b4966d7e6d" (UID: "603cb437-c986-4f41-8153-17b4966d7e6d"). InnerVolumeSpecName "kube-api-access-97brs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 11:50:04.921441 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:50:04.921349 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/603cb437-c986-4f41-8153-17b4966d7e6d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "603cb437-c986-4f41-8153-17b4966d7e6d" (UID: "603cb437-c986-4f41-8153-17b4966d7e6d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 11:50:05.020520 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:50:05.020493 2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/603cb437-c986-4f41-8153-17b4966d7e6d-tls-certs\") on node \"ip-10-0-136-15.ec2.internal\" DevicePath \"\"" Jul 07 11:50:05.020640 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:50:05.020527 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-97brs\" (UniqueName: \"kubernetes.io/projected/603cb437-c986-4f41-8153-17b4966d7e6d-kube-api-access-97brs\") on node \"ip-10-0-136-15.ec2.internal\" DevicePath \"\"" Jul 07 11:50:05.020640 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:50:05.020543 2562 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/603cb437-c986-4f41-8153-17b4966d7e6d-dshm\") on node \"ip-10-0-136-15.ec2.internal\" DevicePath \"\"" Jul 07 11:50:05.020640 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:50:05.020560 2562 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/603cb437-c986-4f41-8153-17b4966d7e6d-tmp-dir\") on node \"ip-10-0-136-15.ec2.internal\" DevicePath \"\"" Jul 07 11:50:05.020640 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:50:05.020572 2562 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/603cb437-c986-4f41-8153-17b4966d7e6d-home\") on node \"ip-10-0-136-15.ec2.internal\" DevicePath \"\"" Jul 07 11:50:05.224496 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:50:05.224465 2562 generic.go:356] "Generic (PLEG): container finished" podID="603cb437-c986-4f41-8153-17b4966d7e6d" containerID="7c83bfb1461708d62df428717a12aacfa5cea193a3b658449538beec96540ab8" exitCode=0 Jul 07 11:50:05.224667 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:50:05.224509 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-43ebe2/sub-test-model-43ebe2-kserve-848ff5cbdf-wjkqw" Jul 07 11:50:05.224667 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:50:05.224517 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-43ebe2/sub-test-model-43ebe2-kserve-848ff5cbdf-wjkqw" event={"ID":"603cb437-c986-4f41-8153-17b4966d7e6d","Type":"ContainerDied","Data":"7c83bfb1461708d62df428717a12aacfa5cea193a3b658449538beec96540ab8"} Jul 07 11:50:05.224667 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:50:05.224548 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-43ebe2/sub-test-model-43ebe2-kserve-848ff5cbdf-wjkqw" event={"ID":"603cb437-c986-4f41-8153-17b4966d7e6d","Type":"ContainerDied","Data":"96db287440658a3d804bdf80acb045707456988fc8e8c22aab25b297fe8ecce0"} Jul 07 11:50:05.224667 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:50:05.224566 2562 scope.go:117] "RemoveContainer" containerID="7c83bfb1461708d62df428717a12aacfa5cea193a3b658449538beec96540ab8" Jul 07 11:50:05.234120 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:50:05.234106 2562 scope.go:117] "RemoveContainer" containerID="7c83bfb1461708d62df428717a12aacfa5cea193a3b658449538beec96540ab8" Jul 07 11:50:05.234354 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:50:05.234336 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7c83bfb1461708d62df428717a12aacfa5cea193a3b658449538beec96540ab8\": container with ID starting with 7c83bfb1461708d62df428717a12aacfa5cea193a3b658449538beec96540ab8 not found: ID does not exist" containerID="7c83bfb1461708d62df428717a12aacfa5cea193a3b658449538beec96540ab8" Jul 07 11:50:05.234408 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:50:05.234363 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7c83bfb1461708d62df428717a12aacfa5cea193a3b658449538beec96540ab8"} err="failed to get container status \"7c83bfb1461708d62df428717a12aacfa5cea193a3b658449538beec96540ab8\": rpc error: code = NotFound desc = could not find container \"7c83bfb1461708d62df428717a12aacfa5cea193a3b658449538beec96540ab8\": container with ID starting with 7c83bfb1461708d62df428717a12aacfa5cea193a3b658449538beec96540ab8 not found: ID does not exist" Jul 07 11:50:05.246749 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:50:05.246728 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-43ebe2/sub-test-model-43ebe2-kserve-848ff5cbdf-wjkqw"] Jul 07 11:50:05.250404 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:50:05.250384 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-43ebe2/sub-test-model-43ebe2-kserve-848ff5cbdf-wjkqw"] Jul 07 11:50:05.767016 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:50:05.766984 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="603cb437-c986-4f41-8153-17b4966d7e6d" path="/var/lib/kubelet/pods/603cb437-c986-4f41-8153-17b4966d7e6d/volumes" Jul 07 11:50:14.661653 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:50:14.661619 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:50:14.758043 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:50:14.758018 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:50:15.757359 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:50:15.757323 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:50:20.013234 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:50:20.013140 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sns5p"] Jul 07 11:50:37.247563 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:50:37.247530 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:50:37.348039 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:50:37.348011 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:50:37.658928 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:50:37.658883 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sns5p"] Jul 07 11:50:44.331332 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:50:44.331291 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:50:44.470421 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:50:44.470390 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:50:46.145793 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:50:46.145756 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:50:46.406085 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:50:46.406005 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:50:51.550362 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:50:51.550328 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sns5p"] Jul 07 11:51:05.169690 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:51:05.169655 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sns5p"] Jul 07 11:51:08.932750 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:51:08.932715 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:51:09.813581 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:51:09.813544 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:51:16.240595 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:51:16.240557 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tzf62_d2340b4b-179a-4c0a-9879-13b49897c939/ovn-acl-logging/0.log" Jul 07 11:51:16.245489 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:51:16.245471 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tzf62_d2340b4b-179a-4c0a-9879-13b49897c939/ovn-acl-logging/0.log" Jul 07 11:51:16.258442 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:51:16.258418 2562 scope.go:117] "RemoveContainer" containerID="ce07740849e104b387923d94904290576b1b993f940193e7c4be456777762bed" Jul 07 11:51:19.046063 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:51:19.046029 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sns5p"] Jul 07 11:51:35.094553 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:51:35.094512 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:52:18.461377 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:52:18.461340 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:52:18.887290 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:52:18.887254 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:52:19.111671 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:52:19.111631 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:52:19.207882 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:52:19.207790 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:52:19.421622 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:52:19.421589 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:52:22.860102 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:52:22.860066 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sns5p"] Jul 07 11:52:25.740422 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:52:25.740387 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:52:25.842662 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:52:25.842619 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:52:26.032641 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:52:26.032539 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:52:26.131078 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:52:26.131020 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:52:26.270571 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:52:26.270533 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:52:26.461174 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:52:26.461137 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:52:26.848491 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:52:26.848404 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:52:26.952495 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:52:26.952468 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:52:27.054334 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:52:27.054302 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:52:27.181620 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:52:27.181591 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-43ebe2-5d67f4d78d-d7hhc"] Jul 07 11:52:27.181821 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:52:27.181797 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-shared-b-43ebe2-5d67f4d78d-d7hhc" podUID="fa8dff7c-c144-4218-83bc-204ee7ea12e3" containerName="maas-api" containerID="cri-o://094efed1e97281203624a0cb76d612ce46face0a00ba5f07283bf799a1d20543" gracePeriod=30 Jul 07 11:52:27.257800 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:52:27.257768 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:52:27.423773 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:52:27.423748 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-43ebe2-5d67f4d78d-d7hhc" Jul 07 11:52:27.566178 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:52:27.566107 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fa8dff7c-c144-4218-83bc-204ee7ea12e3-openshift-service-ca\") pod \"fa8dff7c-c144-4218-83bc-204ee7ea12e3\" (UID: \"fa8dff7c-c144-4218-83bc-204ee7ea12e3\") " Jul 07 11:52:27.566178 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:52:27.566147 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b94jq\" (UniqueName: \"kubernetes.io/projected/fa8dff7c-c144-4218-83bc-204ee7ea12e3-kube-api-access-b94jq\") pod \"fa8dff7c-c144-4218-83bc-204ee7ea12e3\" (UID: \"fa8dff7c-c144-4218-83bc-204ee7ea12e3\") " Jul 07 11:52:27.566334 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:52:27.566210 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fa8dff7c-c144-4218-83bc-204ee7ea12e3-maas-api-tls\") pod \"fa8dff7c-c144-4218-83bc-204ee7ea12e3\" (UID: \"fa8dff7c-c144-4218-83bc-204ee7ea12e3\") " Jul 07 11:52:27.566471 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:52:27.566450 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fa8dff7c-c144-4218-83bc-204ee7ea12e3-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "fa8dff7c-c144-4218-83bc-204ee7ea12e3" (UID: "fa8dff7c-c144-4218-83bc-204ee7ea12e3"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 11:52:27.568332 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:52:27.568305 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fa8dff7c-c144-4218-83bc-204ee7ea12e3-kube-api-access-b94jq" (OuterVolumeSpecName: "kube-api-access-b94jq") pod "fa8dff7c-c144-4218-83bc-204ee7ea12e3" (UID: "fa8dff7c-c144-4218-83bc-204ee7ea12e3"). InnerVolumeSpecName "kube-api-access-b94jq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 11:52:27.568444 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:52:27.568364 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fa8dff7c-c144-4218-83bc-204ee7ea12e3-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "fa8dff7c-c144-4218-83bc-204ee7ea12e3" (UID: "fa8dff7c-c144-4218-83bc-204ee7ea12e3"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 11:52:27.666799 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:52:27.666774 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b94jq\" (UniqueName: \"kubernetes.io/projected/fa8dff7c-c144-4218-83bc-204ee7ea12e3-kube-api-access-b94jq\") on node \"ip-10-0-136-15.ec2.internal\" DevicePath \"\"" Jul 07 11:52:27.666799 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:52:27.666797 2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fa8dff7c-c144-4218-83bc-204ee7ea12e3-maas-api-tls\") on node \"ip-10-0-136-15.ec2.internal\" DevicePath \"\"" Jul 07 11:52:27.666941 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:52:27.666806 2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fa8dff7c-c144-4218-83bc-204ee7ea12e3-openshift-service-ca\") on node \"ip-10-0-136-15.ec2.internal\" DevicePath \"\"" Jul 07 11:52:27.906327 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:52:27.906297 2562 generic.go:356] "Generic (PLEG): container finished" podID="fa8dff7c-c144-4218-83bc-204ee7ea12e3" containerID="094efed1e97281203624a0cb76d612ce46face0a00ba5f07283bf799a1d20543" exitCode=0 Jul 07 11:52:27.906646 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:52:27.906342 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-43ebe2-5d67f4d78d-d7hhc" Jul 07 11:52:27.906646 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:52:27.906380 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-43ebe2-5d67f4d78d-d7hhc" event={"ID":"fa8dff7c-c144-4218-83bc-204ee7ea12e3","Type":"ContainerDied","Data":"094efed1e97281203624a0cb76d612ce46face0a00ba5f07283bf799a1d20543"} Jul 07 11:52:27.906646 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:52:27.906439 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-43ebe2-5d67f4d78d-d7hhc" event={"ID":"fa8dff7c-c144-4218-83bc-204ee7ea12e3","Type":"ContainerDied","Data":"13c71b1966e02519fd1cbb437fd21c4e64140d5581fb4ee80c59f0facfa99ec8"} Jul 07 11:52:27.906646 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:52:27.906463 2562 scope.go:117] "RemoveContainer" containerID="094efed1e97281203624a0cb76d612ce46face0a00ba5f07283bf799a1d20543" Jul 07 11:52:27.915224 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:52:27.915207 2562 scope.go:117] "RemoveContainer" containerID="094efed1e97281203624a0cb76d612ce46face0a00ba5f07283bf799a1d20543" Jul 07 11:52:27.915472 ip-10-0-136-15 kubenswrapper[2562]: E0707 11:52:27.915457 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"094efed1e97281203624a0cb76d612ce46face0a00ba5f07283bf799a1d20543\": container with ID starting with 094efed1e97281203624a0cb76d612ce46face0a00ba5f07283bf799a1d20543 not found: ID does not exist" containerID="094efed1e97281203624a0cb76d612ce46face0a00ba5f07283bf799a1d20543" Jul 07 11:52:27.915518 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:52:27.915479 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"094efed1e97281203624a0cb76d612ce46face0a00ba5f07283bf799a1d20543"} err="failed to get container status \"094efed1e97281203624a0cb76d612ce46face0a00ba5f07283bf799a1d20543\": rpc error: code = NotFound desc = could not find container \"094efed1e97281203624a0cb76d612ce46face0a00ba5f07283bf799a1d20543\": container with ID starting with 094efed1e97281203624a0cb76d612ce46face0a00ba5f07283bf799a1d20543 not found: ID does not exist" Jul 07 11:52:27.927848 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:52:27.927828 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-43ebe2-5d67f4d78d-d7hhc"] Jul 07 11:52:27.932883 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:52:27.932862 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-43ebe2-5d67f4d78d-d7hhc"] Jul 07 11:52:29.766796 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:52:29.766762 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fa8dff7c-c144-4218-83bc-204ee7ea12e3" path="/var/lib/kubelet/pods/fa8dff7c-c144-4218-83bc-204ee7ea12e3/volumes" Jul 07 11:52:31.653830 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:52:31.653791 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-sns5p"] Jul 07 11:52:33.352038 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:52:33.352003 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:52:33.444102 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:52:33.444074 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:52:33.624806 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:52:33.624774 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:52:33.846811 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:52:33.846774 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:52:39.508073 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:52:39.508044 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29723745-pxrlh_bb90e11a-28dc-4bb8-a28c-d19d8272e6e1/cleanup/0.log" Jul 07 11:52:40.205727 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:52:40.205693 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6dc7dd79cd-wxb6p_8ccc2735-03a2-45ed-94c6-035b5288903c/copy-manifests/0.log" Jul 07 11:52:40.218568 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:52:40.218545 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6dc7dd79cd-wxb6p_8ccc2735-03a2-45ed-94c6-035b5288903c/manager/0.log" Jul 07 11:52:42.082539 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:52:42.082514 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-rwrbw_8557e6db-fe7e-47a1-9700-da25802680eb/kuadrant-console-plugin/0.log" Jul 07 11:52:42.425008 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:52:42.424980 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-sns5p_de71dd9a-d584-4c85-b592-4381ce0a591d/limitador/0.log" Jul 07 11:52:43.109739 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:52:43.109710 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5df5f4cc47-4ckp9_f0e9dcfd-d744-4b75-8186-bbbd8c43bad2/kube-auth-proxy/0.log" Jul 07 11:52:44.031164 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:52:44.031138 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-ncx5l_c21c6534-63ce-4930-8053-a59ae7cc12be/main/0.log" Jul 07 11:52:44.143328 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:52:44.143302 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-z5jvw_1199bebe-3056-49d7-bb3d-1e5dcf5c728f/main/0.log" Jul 07 11:52:48.682999 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:52:48.682964 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:52:48.794095 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:52:48.794060 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" limit=1024 Jul 07 11:52:51.496736 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:52:51.496704 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-7ml6d_a4b1e5f7-71c6-4349-8dad-ef5763f9bfb9/global-pull-secret-syncer/0.log" Jul 07 11:52:51.662180 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:52:51.662152 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-4b5qw_663d4847-eafe-43d6-a4b8-2bec15919119/konnectivity-agent/0.log" Jul 07 11:52:51.748230 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:52:51.748158 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-136-15.ec2.internal_a0c235cfc97783f979ae1f6fa1e22e58/haproxy/0.log" Jul 07 11:52:56.700413 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:52:56.700381 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-rwrbw_8557e6db-fe7e-47a1-9700-da25802680eb/kuadrant-console-plugin/0.log" Jul 07 11:52:56.989875 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:52:56.989800 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-sns5p_de71dd9a-d584-4c85-b592-4381ce0a591d/limitador/0.log" Jul 07 11:52:58.582748 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:52:58.582672 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_9799c267-e325-4239-94d3-023d50dd3fb3/alertmanager/0.log" Jul 07 11:52:58.615867 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:52:58.615842 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_9799c267-e325-4239-94d3-023d50dd3fb3/config-reloader/0.log" Jul 07 11:52:58.653210 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:52:58.653184 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_9799c267-e325-4239-94d3-023d50dd3fb3/kube-rbac-proxy-web/0.log" Jul 07 11:52:58.681518 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:52:58.681494 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_9799c267-e325-4239-94d3-023d50dd3fb3/kube-rbac-proxy/0.log" Jul 07 11:52:58.718239 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:52:58.718208 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_9799c267-e325-4239-94d3-023d50dd3fb3/kube-rbac-proxy-metric/0.log" Jul 07 11:52:58.756743 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:52:58.756720 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_9799c267-e325-4239-94d3-023d50dd3fb3/prom-label-proxy/0.log" Jul 07 11:52:58.797477 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:52:58.797455 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_9799c267-e325-4239-94d3-023d50dd3fb3/init-config-reloader/0.log" Jul 07 11:52:58.905095 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:52:58.905073 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-lvbqx_124f8526-2fa2-4206-817b-85d6b0d00d56/kube-state-metrics/0.log" Jul 07 11:52:58.941726 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:52:58.941705 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-lvbqx_124f8526-2fa2-4206-817b-85d6b0d00d56/kube-rbac-proxy-main/0.log" Jul 07 11:52:58.973172 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:52:58.973153 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-lvbqx_124f8526-2fa2-4206-817b-85d6b0d00d56/kube-rbac-proxy-self/0.log" Jul 07 11:52:59.011886 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:52:59.011867 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-8654d65bc6-c676s_58c40425-d108-438f-8e91-4af50c22b58c/metrics-server/0.log" Jul 07 11:52:59.045247 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:52:59.045224 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-d5dcff7cf-qhxlp_7f341820-bee0-4ac7-964f-4a20ca2eb4f4/monitoring-plugin/0.log" Jul 07 11:52:59.091224 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:52:59.091208 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-2ffb8_acd61036-c152-4340-9a8a-cb48eff0f398/node-exporter/0.log" Jul 07 11:52:59.166050 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:52:59.166002 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-2ffb8_acd61036-c152-4340-9a8a-cb48eff0f398/kube-rbac-proxy/0.log" Jul 07 11:52:59.264594 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:52:59.264575 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-2ffb8_acd61036-c152-4340-9a8a-cb48eff0f398/init-textfile/0.log" Jul 07 11:52:59.604477 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:52:59.604453 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-kqpvf_2ac7717f-0d38-4ac8-b705-d6b9bcbf7859/kube-rbac-proxy-main/0.log" Jul 07 11:52:59.639692 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:52:59.639674 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-kqpvf_2ac7717f-0d38-4ac8-b705-d6b9bcbf7859/kube-rbac-proxy-self/0.log" Jul 07 11:52:59.673364 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:52:59.673348 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-kqpvf_2ac7717f-0d38-4ac8-b705-d6b9bcbf7859/openshift-state-metrics/0.log" Jul 07 11:52:59.713474 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:52:59.713452 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_67e0c033-c10d-4952-883c-56e13a8f2e23/prometheus/0.log" Jul 07 11:52:59.775688 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:52:59.775619 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_67e0c033-c10d-4952-883c-56e13a8f2e23/config-reloader/0.log" Jul 07 11:52:59.839668 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:52:59.839647 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_67e0c033-c10d-4952-883c-56e13a8f2e23/thanos-sidecar/0.log" Jul 07 11:52:59.908848 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:52:59.908828 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_67e0c033-c10d-4952-883c-56e13a8f2e23/kube-rbac-proxy-web/0.log" Jul 07 11:52:59.977369 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:52:59.977352 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_67e0c033-c10d-4952-883c-56e13a8f2e23/kube-rbac-proxy/0.log" Jul 07 11:53:00.025854 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:53:00.025807 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_67e0c033-c10d-4952-883c-56e13a8f2e23/kube-rbac-proxy-thanos/0.log" Jul 07 11:53:00.053107 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:53:00.053090 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_67e0c033-c10d-4952-883c-56e13a8f2e23/init-config-reloader/0.log" Jul 07 11:53:00.089972 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:53:00.089954 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-rpmtl_5bdca415-9aba-484a-9d71-fe763b0e6215/prometheus-operator/0.log" Jul 07 11:53:00.113395 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:53:00.113377 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-rpmtl_5bdca415-9aba-484a-9d71-fe763b0e6215/kube-rbac-proxy/0.log" Jul 07 11:53:00.175109 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:53:00.175090 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5f756946cf-8fnhr_0eea7695-f8a3-4f8e-aea5-f619a6e06fcb/telemeter-client/0.log" Jul 07 11:53:00.197674 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:53:00.197655 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5f756946cf-8fnhr_0eea7695-f8a3-4f8e-aea5-f619a6e06fcb/reload/0.log" Jul 07 11:53:00.219632 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:53:00.219606 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5f756946cf-8fnhr_0eea7695-f8a3-4f8e-aea5-f619a6e06fcb/kube-rbac-proxy/0.log" Jul 07 11:53:00.253262 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:53:00.253242 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-746f69848d-65tjq_3e27d653-3f08-4102-b75c-ac10382e1959/thanos-query/0.log" Jul 07 11:53:00.276704 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:53:00.276638 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-746f69848d-65tjq_3e27d653-3f08-4102-b75c-ac10382e1959/kube-rbac-proxy-web/0.log" Jul 07 11:53:00.299264 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:53:00.299246 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-746f69848d-65tjq_3e27d653-3f08-4102-b75c-ac10382e1959/kube-rbac-proxy/0.log" Jul 07 11:53:00.322844 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:53:00.322816 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-746f69848d-65tjq_3e27d653-3f08-4102-b75c-ac10382e1959/prom-label-proxy/0.log" Jul 07 11:53:00.350598 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:53:00.350580 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-746f69848d-65tjq_3e27d653-3f08-4102-b75c-ac10382e1959/kube-rbac-proxy-rules/0.log" Jul 07 11:53:00.383595 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:53:00.383576 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-746f69848d-65tjq_3e27d653-3f08-4102-b75c-ac10382e1959/kube-rbac-proxy-metrics/0.log" Jul 07 11:53:00.842950 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:53:00.842910 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-rl7lp/perf-node-gather-daemonset-xhkth"] Jul 07 11:53:00.843536 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:53:00.843438 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="603cb437-c986-4f41-8153-17b4966d7e6d" containerName="main" Jul 07 11:53:00.843536 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:53:00.843463 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="603cb437-c986-4f41-8153-17b4966d7e6d" containerName="main" Jul 07 11:53:00.843536 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:53:00.843485 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fa8dff7c-c144-4218-83bc-204ee7ea12e3" containerName="maas-api" Jul 07 11:53:00.843536 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:53:00.843494 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="fa8dff7c-c144-4218-83bc-204ee7ea12e3" containerName="maas-api" Jul 07 11:53:00.843862 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:53:00.843597 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="fa8dff7c-c144-4218-83bc-204ee7ea12e3" containerName="maas-api" Jul 07 11:53:00.843862 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:53:00.843614 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="603cb437-c986-4f41-8153-17b4966d7e6d" containerName="main" Jul 07 11:53:00.849805 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:53:00.848631 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-rl7lp/perf-node-gather-daemonset-xhkth" Jul 07 11:53:00.852024 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:53:00.852005 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-rl7lp\"/\"default-dockercfg-2t97q\"" Jul 07 11:53:00.852130 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:53:00.852014 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-rl7lp\"/\"kube-root-ca.crt\"" Jul 07 11:53:00.852130 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:53:00.852071 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-rl7lp\"/\"openshift-service-ca.crt\"" Jul 07 11:53:00.864212 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:53:00.864191 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-rl7lp/perf-node-gather-daemonset-xhkth"] Jul 07 11:53:00.958976 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:53:00.958950 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3af1434c-d952-4111-a3c2-43ca790d75d0-lib-modules\") pod \"perf-node-gather-daemonset-xhkth\" (UID: \"3af1434c-d952-4111-a3c2-43ca790d75d0\") " pod="openshift-must-gather-rl7lp/perf-node-gather-daemonset-xhkth" Jul 07 11:53:00.959113 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:53:00.958991 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/3af1434c-d952-4111-a3c2-43ca790d75d0-proc\") pod \"perf-node-gather-daemonset-xhkth\" (UID: \"3af1434c-d952-4111-a3c2-43ca790d75d0\") " pod="openshift-must-gather-rl7lp/perf-node-gather-daemonset-xhkth" Jul 07 11:53:00.959113 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:53:00.959014 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/3af1434c-d952-4111-a3c2-43ca790d75d0-podres\") pod \"perf-node-gather-daemonset-xhkth\" (UID: \"3af1434c-d952-4111-a3c2-43ca790d75d0\") " pod="openshift-must-gather-rl7lp/perf-node-gather-daemonset-xhkth" Jul 07 11:53:00.959113 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:53:00.959087 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3af1434c-d952-4111-a3c2-43ca790d75d0-sys\") pod \"perf-node-gather-daemonset-xhkth\" (UID: \"3af1434c-d952-4111-a3c2-43ca790d75d0\") " pod="openshift-must-gather-rl7lp/perf-node-gather-daemonset-xhkth" Jul 07 11:53:00.959238 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:53:00.959160 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hdpg5\" (UniqueName: \"kubernetes.io/projected/3af1434c-d952-4111-a3c2-43ca790d75d0-kube-api-access-hdpg5\") pod \"perf-node-gather-daemonset-xhkth\" (UID: \"3af1434c-d952-4111-a3c2-43ca790d75d0\") " pod="openshift-must-gather-rl7lp/perf-node-gather-daemonset-xhkth" Jul 07 11:53:01.060381 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:53:01.060344 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3af1434c-d952-4111-a3c2-43ca790d75d0-sys\") pod \"perf-node-gather-daemonset-xhkth\" (UID: \"3af1434c-d952-4111-a3c2-43ca790d75d0\") " pod="openshift-must-gather-rl7lp/perf-node-gather-daemonset-xhkth" Jul 07 11:53:01.060583 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:53:01.060397 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hdpg5\" (UniqueName: \"kubernetes.io/projected/3af1434c-d952-4111-a3c2-43ca790d75d0-kube-api-access-hdpg5\") pod \"perf-node-gather-daemonset-xhkth\" (UID: \"3af1434c-d952-4111-a3c2-43ca790d75d0\") " pod="openshift-must-gather-rl7lp/perf-node-gather-daemonset-xhkth" Jul 07 11:53:01.060583 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:53:01.060470 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3af1434c-d952-4111-a3c2-43ca790d75d0-sys\") pod \"perf-node-gather-daemonset-xhkth\" (UID: \"3af1434c-d952-4111-a3c2-43ca790d75d0\") " pod="openshift-must-gather-rl7lp/perf-node-gather-daemonset-xhkth" Jul 07 11:53:01.060583 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:53:01.060512 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3af1434c-d952-4111-a3c2-43ca790d75d0-lib-modules\") pod \"perf-node-gather-daemonset-xhkth\" (UID: \"3af1434c-d952-4111-a3c2-43ca790d75d0\") " pod="openshift-must-gather-rl7lp/perf-node-gather-daemonset-xhkth" Jul 07 11:53:01.060583 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:53:01.060551 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/3af1434c-d952-4111-a3c2-43ca790d75d0-proc\") pod \"perf-node-gather-daemonset-xhkth\" (UID: \"3af1434c-d952-4111-a3c2-43ca790d75d0\") " pod="openshift-must-gather-rl7lp/perf-node-gather-daemonset-xhkth" Jul 07 11:53:01.060792 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:53:01.060588 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/3af1434c-d952-4111-a3c2-43ca790d75d0-podres\") pod \"perf-node-gather-daemonset-xhkth\" (UID: \"3af1434c-d952-4111-a3c2-43ca790d75d0\") " pod="openshift-must-gather-rl7lp/perf-node-gather-daemonset-xhkth" Jul 07 11:53:01.060792 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:53:01.060636 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3af1434c-d952-4111-a3c2-43ca790d75d0-lib-modules\") pod \"perf-node-gather-daemonset-xhkth\" (UID: \"3af1434c-d952-4111-a3c2-43ca790d75d0\") " pod="openshift-must-gather-rl7lp/perf-node-gather-daemonset-xhkth" Jul 07 11:53:01.060792 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:53:01.060636 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/3af1434c-d952-4111-a3c2-43ca790d75d0-proc\") pod \"perf-node-gather-daemonset-xhkth\" (UID: \"3af1434c-d952-4111-a3c2-43ca790d75d0\") " pod="openshift-must-gather-rl7lp/perf-node-gather-daemonset-xhkth" Jul 07 11:53:01.060792 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:53:01.060727 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/3af1434c-d952-4111-a3c2-43ca790d75d0-podres\") pod \"perf-node-gather-daemonset-xhkth\" (UID: \"3af1434c-d952-4111-a3c2-43ca790d75d0\") " pod="openshift-must-gather-rl7lp/perf-node-gather-daemonset-xhkth" Jul 07 11:53:01.070793 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:53:01.070772 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hdpg5\" (UniqueName: \"kubernetes.io/projected/3af1434c-d952-4111-a3c2-43ca790d75d0-kube-api-access-hdpg5\") pod \"perf-node-gather-daemonset-xhkth\" (UID: \"3af1434c-d952-4111-a3c2-43ca790d75d0\") " pod="openshift-must-gather-rl7lp/perf-node-gather-daemonset-xhkth" Jul 07 11:53:01.161391 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:53:01.161366 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-rl7lp/perf-node-gather-daemonset-xhkth" Jul 07 11:53:01.288668 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:53:01.288640 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-rl7lp/perf-node-gather-daemonset-xhkth"] Jul 07 11:53:01.290487 ip-10-0-136-15 kubenswrapper[2562]: W0707 11:53:01.290462 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod3af1434c_d952_4111_a3c2_43ca790d75d0.slice/crio-1303d00c03d017c36a91e5a093dba4c9a5aded927c7326556e5f67b5becef1c4 WatchSource:0}: Error finding container 1303d00c03d017c36a91e5a093dba4c9a5aded927c7326556e5f67b5becef1c4: Status 404 returned error can't find the container with id 1303d00c03d017c36a91e5a093dba4c9a5aded927c7326556e5f67b5becef1c4 Jul 07 11:53:02.065149 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:53:02.065110 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-rl7lp/perf-node-gather-daemonset-xhkth" event={"ID":"3af1434c-d952-4111-a3c2-43ca790d75d0","Type":"ContainerStarted","Data":"bce482588fc8fef53584f96c2b28490ae19739a8a93321a10ba33730cfde047c"} Jul 07 11:53:02.065547 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:53:02.065171 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-rl7lp/perf-node-gather-daemonset-xhkth" event={"ID":"3af1434c-d952-4111-a3c2-43ca790d75d0","Type":"ContainerStarted","Data":"1303d00c03d017c36a91e5a093dba4c9a5aded927c7326556e5f67b5becef1c4"} Jul 07 11:53:02.065547 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:53:02.065198 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-rl7lp/perf-node-gather-daemonset-xhkth" Jul 07 11:53:02.083720 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:53:02.083669 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-rl7lp/perf-node-gather-daemonset-xhkth" podStartSLOduration=2.083650181 podStartE2EDuration="2.083650181s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 11:53:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 11:53:02.081805856 +0000 UTC m=+2506.495675021" watchObservedRunningTime="2026-07-07 11:53:02.083650181 +0000 UTC m=+2506.497519350" Jul 07 11:53:02.642542 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:53:02.642506 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-685bfb5f64-428ms_a40bc7e6-49a7-49f7-b204-e03679e7f1fc/download-server/0.log" Jul 07 11:53:04.346240 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:53:04.346214 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-k8wv9_c5da13ef-838f-4deb-9b49-7b12cab2bb84/dns/0.log" Jul 07 11:53:04.370301 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:53:04.370272 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-k8wv9_c5da13ef-838f-4deb-9b49-7b12cab2bb84/kube-rbac-proxy/0.log" Jul 07 11:53:04.395927 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:53:04.395890 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-77v22_a9da5840-e6b4-4719-98cd-ef0d46d8470c/dns-node-resolver/0.log" Jul 07 11:53:05.014500 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:53:05.014471 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-528n2_e93d8868-ca25-4c0a-9f25-da7b8c286f60/node-ca/0.log" Jul 07 11:53:06.296047 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:53:06.296021 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5df5f4cc47-4ckp9_f0e9dcfd-d744-4b75-8186-bbbd8c43bad2/kube-auth-proxy/0.log" Jul 07 11:53:07.360734 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:53:07.360704 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-mmtwk_0bcfaddb-0c6d-4519-a1a7-c23b3a610272/serve-healthcheck-canary/0.log" Jul 07 11:53:07.963415 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:53:07.963336 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-5279v_267a6b3c-8130-497b-b850-fca340df911a/kube-rbac-proxy/0.log" Jul 07 11:53:07.987642 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:53:07.987617 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-5279v_267a6b3c-8130-497b-b850-fca340df911a/exporter/0.log" Jul 07 11:53:08.020580 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:53:08.020549 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-5279v_267a6b3c-8130-497b-b850-fca340df911a/extractor/0.log" Jul 07 11:53:08.080871 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:53:08.080848 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-rl7lp/perf-node-gather-daemonset-xhkth" Jul 07 11:53:10.886684 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:53:10.886649 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29723745-pxrlh_bb90e11a-28dc-4bb8-a28c-d19d8272e6e1/cleanup/0.log" Jul 07 11:53:11.278379 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:53:11.278351 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6dc7dd79cd-wxb6p_8ccc2735-03a2-45ed-94c6-035b5288903c/manager/0.log" Jul 07 11:53:11.297355 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:53:11.297334 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6dc7dd79cd-wxb6p_8ccc2735-03a2-45ed-94c6-035b5288903c/copy-manifests/0.log" Jul 07 11:53:11.564985 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:53:11.564888 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5df5f4cc47-4ckp9_f0e9dcfd-d744-4b75-8186-bbbd8c43bad2/kube-auth-proxy/0.log" Jul 07 11:53:12.872889 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:53:12.872804 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-7769b86749-cdjb9_eb008980-137b-4f0f-b31c-b1c5db989c50/manager/0.log" Jul 07 11:53:13.037641 ip-10-0-136-15 kubenswrapper[2562]: I0707 11:53:13.037611 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5df5f4cc47-4ckp9_f0e9dcfd-d744-4b75-8186-bbbd8c43bad2/kube-auth-proxy/0.log"